unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / Atom feed
* Connection issues between nginx and unicorn
@ 2020-08-18 15:15 Michael Bernstein
  2020-08-18 18:06 ` Eric Wong
  0 siblings, 1 reply; 3+ messages in thread
From: Michael Bernstein @ 2020-08-18 15:15 UTC (permalink / raw)
  To: unicorn-public; +Cc: Daniel Libanori, Luiz Ferreira

Hello,

We are having a lot of connection issues between nginx and unicorn,
but we don't know exactly where is the issue. Can somebody help us?

Exemple from our nginx logs:
[error] 6#6: *4269348 connect() failed (111: Connection refused) while
connecting to upstream, client: 10.2.2.252, server: _, request: "POST
/api/v1/documents?access_token=....
[error] 6#6: *4269169 recv() failed (104: Connection reset by peer)
while reading response header from upstream, client: 10.2.10.120,
server: _, request: "GET /api/v1/documents/.....
[error] 6#6: *4268896 upstream timed out (110: Operation timed out)
while connecting to upstream, client: 10.2.2.252, server: _, request:
"GET /accounts/.... HTTP/1.1", upstream:
"http://172.20.214.176:80/accounts...", host: "app.clicksign.com",
referrer: "https://app.clicksign.com/..."

############
Unicorn configuration file:

# frozen_string_literal: true

listen 8080
worker_processes 4
timeout 60

preload_app true
GC.respond_to?(:copy_on_write_friendly=) &&
  (GC.copy_on_write_friendly = true)

before_fork do |server, worker|
  Signal.trap 'TERM' do
    puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
    Process.kill 'QUIT', Process.pid
  end

  defined?(ActiveRecord::Base) &&
    ActiveRecord::Base.connection.disconnect!

  old_pid = "#{server.config[:pid]}.oldbin"
  if old_pid != server.pid
    begin
      sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
      Process.kill(sig, File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH
    end
  end

  sleep 1
end

after_fork do |_server, _worker|
  Signal.trap 'TERM' do
    puts 'Unicorn worker intercepting TERM and doing nothing. Wait for
master to send QUIT'
  end

  defined?(ActiveRecord::Base) &&
    ActiveRecord::Base.establish_connection
end


############
nginx configuration file:

proxy_cache_path /var/cache/nginx/ levels=1:2 keys_zone=assets_cache:10m
                 max_size=1g inactive=60m use_temp_path=off;

resolver kube-dns.kube-system.svc.cluster.local valid=5s;

upstream backend {
  server unicorn;
}

server {
  server_name _;

  server_tokens off;
  keepalive_timeout 5;
  client_max_body_size 100M;
  client_body_buffer_size 30M;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay off;

  gzip on;
  gzip_min_length 1000;
  gzip_proxied any;
  gzip_buffers 16 8k;
  gzip_http_version 1.0;
  gzip_types text/css
             text/plain
             text/javascript
             application/javascript
             application/json
             application/x-javascript
             application/xml
             application/xml+rss
             application/xhtml+xml
             application/x-font-ttf
             application/x-font-opentype
             application/vnd.ms-fontobject
             image/svg+xml
             image/x-icon
             application/rss+xml
             application/atom_xml;

  location /healthcheck {
    return 204;
  }

  location ~* ^/(fonts|webfonts|assets|packs) {
    proxy_redirect off;
    proxy_cache assets_cache;
    proxy_cache_valid 120m;
    proxy_pass http://backend;
    add_header Cache-Control public;
    if ($http_origin ~
'^((https*?:\/\/).*?((\.clicksign)\.(com|me|dev)))($|\/.*$)$' ){
      add_header Access-Control-Allow-Origin $http_origin;
    }
    expires max;
  }

  location / {
    proxy_pass http://backend;
    proxy_redirect off;
    proxy_read_timeout 60;
    proxy_set_header Host $http_host;
  }
}

######

Thanks,

Michael Belfer Bernstein
michael.bernstein@clicksign.com
https://www.clicksign.com | https://www.fluxia.com.br

-- 
A informação contida nesta mensagem e seus anexos é considerada secreta, 
para uso exclusivo de seu destinatário. Caso você não seja o destinatário, 
notifique o remetente e elimine esta mensagem. The information contained in 
this message and respective attachments is secret, to be used exclusively 
by its addressee. If you are not the intended addressee, please notify the 
sender and delete this message.

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Connection issues between nginx and unicorn
  2020-08-18 15:15 Connection issues between nginx and unicorn Michael Bernstein
@ 2020-08-18 18:06 ` Eric Wong
  2020-08-22  1:27   ` Eric Wong
  0 siblings, 1 reply; 3+ messages in thread
From: Eric Wong @ 2020-08-18 18:06 UTC (permalink / raw)
  To: Michael Bernstein; +Cc: unicorn-public, Daniel Libanori, Luiz Ferreira

Michael Bernstein <michael.bernstein@clicksign.com> wrote:
> Hello,
> 
> We are having a lot of connection issues between nginx and unicorn,
> but we don't know exactly where is the issue. Can somebody help us?

Sure, replies in line...

> Exemple from our nginx logs:
> [error] 6#6: *4269348 connect() failed (111: Connection refused) while
> connecting to upstream, client: 10.2.2.252, server: _, request: "POST
> /api/v1/documents?access_token=....
> [error] 6#6: *4269169 recv() failed (104: Connection reset by peer)
> while reading response header from upstream, client: 10.2.10.120,
> server: _, request: "GET /api/v1/documents/.....
> [error] 6#6: *4268896 upstream timed out (110: Operation timed out)
> while connecting to upstream, client: 10.2.2.252, server: _, request:
> "GET /accounts/.... HTTP/1.1", upstream:
> "http://172.20.214.176:80/accounts...", host: "app.clicksign.com",
> referrer: "https://app.clicksign.com/..."

Did any connections ever work?

> ############
> Unicorn configuration file:
> 
> # frozen_string_literal: true
> 
> listen 8080

OK, so that means it's listening to all IP addresses on TCP port 8080;
no problem here.

If you use curl to hit 8080 on the host(s) unicorn is running,
it should work.

> worker_processes 4
> timeout 60
> 
> preload_app true
> GC.respond_to?(:copy_on_write_friendly=) &&
>   (GC.copy_on_write_friendly = true)

Off-topic, GC.copy_on_write_friendly is the default in Ruby 2.0+;
(and IIRC it wasn't in any official Ruby release, just the
Phusion "Enterprise Edition")

<snip>

> ############
> nginx configuration file:
> 
> proxy_cache_path /var/cache/nginx/ levels=1:2 keys_zone=assets_cache:10m
>                  max_size=1g inactive=60m use_temp_path=off;
> 
> resolver kube-dns.kube-system.svc.cluster.local valid=5s;
> 
> upstream backend {
>   server unicorn;

Is "unicorn" an entry in your DNS (via search path) or
an entry in /etc/hosts file?

There's also no reference to port 8080 anywhere in your nginx
config, so I don't know how nginx is supposed to know that
unicorn is on port 8080.

I would expect to see something like:

    server unicorn.internal.example.com:8080;

Or if on the same host:

    server 127.0.0.1:8080;

> }
> 
> server {
>   server_name _;

<snip>

>   location ~* ^/(fonts|webfonts|assets|packs) {
>     proxy_redirect off;
>     proxy_cache assets_cache;
>     proxy_cache_valid 120m;
>     proxy_pass http://backend;
>     add_header Cache-Control public;
>     if ($http_origin ~
> '^((https*?:\/\/).*?((\.clicksign)\.(com|me|dev)))($|\/.*$)$' ){
>       add_header Access-Control-Allow-Origin $http_origin;
>     }
>     expires max;
>   }
> 
>   location / {
>     proxy_pass http://backend;
>     proxy_redirect off;
>     proxy_read_timeout 60;
>     proxy_set_header Host $http_host;
>   }

No references to port 8080 in either location block, either.

I don't think you need 8080 in location block if you have them
in the "upstream backend" block; it's been a while since I've
used nginx... But you need 8080 (and the correct IP address or
DNS entry) somewhere.

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Connection issues between nginx and unicorn
  2020-08-18 18:06 ` Eric Wong
@ 2020-08-22  1:27   ` Eric Wong
  0 siblings, 0 replies; 3+ messages in thread
From: Eric Wong @ 2020-08-22  1:27 UTC (permalink / raw)
  To: Michael Bernstein; +Cc: unicorn-public, Daniel Libanori, Luiz Ferreira

Hey, just wondering if things got figured things out, thanks.

cf. https://yhbt.net/unicorn-public/20200818180611.GA18977@dcvr/

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2020-08-22  1:27 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-18 15:15 Connection issues between nginx and unicorn Michael Bernstein
2020-08-18 18:06 ` Eric Wong
2020-08-22  1:27   ` Eric Wong

unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhbt.net/unicorn-public
	git clone --mirror http://ou63pmih66umazou.onion/unicorn-public

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 unicorn-public unicorn-public/ https://yhbt.net/unicorn-public \
		unicorn-public@yhbt.net unicorn-public@bogomips.org mongrel-unicorn@rubyforge.org mongrel-unicorn-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
	public-inbox-index unicorn-public

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
	nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.unicorn
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	unicorn.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git