Those days we almost use cloud for everthing. But sometimes we need to pull files from sftp server. Here are two solutions for that

Pull and remove with sftp

This solution pulls the files then removes them from the remote. There is a gotcha that if you expect a lot of files there might be a chance a file to arrive while the “get -r …” command is executing. Then the “rm *” will remove it. So this is suitable if you expect a few files a week/day

Create a batchfile.sh

get -r upload/* incoming/
rm upload/*

Then add cron

0 5 * * * /usr/bin/sftp -b batchfile.sh username@sftp-corp.company.com

Only pulling with lftp

When I don’t have permissions to remove the files from the remote sftp I use the following off-the-shelf aproach.

This cron is synchronizing files all files to /home/USERNAME/incoming

0 5 * * *  /usr/bin/lftp -u USERNAME,none -e 'mirror --newer-than="now-7days" --only-newer --exclude .ssh --only-missing / /home/USERNAME/incoming; quit' sftp://sftp-corp.company.com

When in your distribution the postgres is stick to version 10 and you have to upgrade to postgres-11 a good way to do a capistrano deploy is like this

Do the system install with

yum install postgresql10-contrib postgresql10-devel

And then in your /shared/.bundle/config add a line showing the location of the pg libraries

---
BUNDLE_PATH: "/opt/application/shared/bundle"
BUNDLE_BUILD__PG: "--with-pg-config=/usr/pgsql-10/bin/pg_config"
BUNDLE_FROZEN: "true"
BUNDLE_JOBS: "4"
BUNDLE_WITHOUT: "development:test"

Thanks to my colleague Kris for finding the solution.

Continue reading

Saboteur by Frederic Moyersoen

Link to the English version:

Continue reading

Continue reading

Continue reading

https://blog.juliobiason.net/thoughts/things-i-learnt-the-hard-way/

Here is how one can setup a nginx to count the https connections made.

Preparation

Create a new folder

mkdir ~/docker_ssl_proxy
cd ~/docker_ssl_proxy

Put a dummy entry in your /etc/hosts file

127.0.0.1 YOURDOMAIN.com

Steps

First generate certificate

openssl req -subj '/CN=YOURDOMAIN.com' -x509 -newkey rsa:4096 -nodes -keyout key.pem -out cert.pem -days 365

create a new file something.conf with the following content

server {
  listen 4000 ssl;
  ssl_certificate /etc/nginx/conf.d/cert.pem;
  ssl_certificate_key /etc/nginx/conf.d/key.pem;

  # access_log /dev/stdout;
  access_log  /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  location / {
      return 200 'With style!';
      add_header Content-Type text/plain;
  }


}

Then run the docker with

docker run --rm -v `pwd`/logs:/var/log/nginx -v `pwd`:/etc/nginx/conf.d -p 4000:4000 nginx

Get the cacert

echo quit | openssl s_client -showcerts -servername server -connect YOURDOMAIN.com:4000 > cacert.pem
curl --cacert cacert.pem https://YOURDOMAIN.com:4000/ -d 'hello world'

And finally do some connections

go-wrk  -c=400 -t=8 -n=10000 -m="POST" -b='{"accountID":"1"}'  -i https://YOURDOMAIN.com:4000

You can do it from the Ruby on Rails application or even better way is to do it where you define your infrastructure and configure your proxy/loadbalancer etc. to allow it.

constrains manage
  class Whitelist
    def matches?(request)
      vpn = IPAddr.new("10.1.0.0/16")
      return true if Rails.env.development? || vpn.include?(request.remote_ip)

      Rails.logger.info("Blocking access for #{request.remote_ip} to #{rifiniti_vpn}")
      false
    end
  end

  constraints Whitelist.new do
    namespace :manage do
	...protected
    end
  end