Sometimes you spend so much time tweaking Redis and queues run in Redis, you forget there are great tools out there to help you out. Let's talk Horizon Run Redis Queues and an issue you could run into as well.

I had Horizon – ^1.2 – up and running with a Forge daemon setup and it did work. It showed it was active, max runtime and max throughput were on default. Under metrics queues I did see default and our own supervisor queue for certificates.
I did however not see our supervisor under the supervisor block on the Dashboard page below the overview. Not on staging anyways.

Checking Queues and Daemons

When I checked for artisan processes I saw:

forge@site-staging:~$ ps aux |grep artisan
forge 7770 0.0 0.1 14856 1092 pts/0 S+ 01:24 0:00 grep --color=auto artisan
forge 24606 0.0 4.2 435312 43364 ? S Mar20 0:21 php artisan queue:work --queue=ssl-manager --sleep=3 --tries=3 -vvv
forge 28312 0.0 4.3 356520 44044 ? S Mar20 0:49 php artisan horizon

This while the supervisor supervisor-1 did show on production. And there much more was up and running as more queues had been started up there.

forge@site-prod-w-1:~$ ps aux |grep artisan
forge 15340 0.0 2.1 356520 43992 ? S Mar18 3:11 php artisan horizon
forge 15347 0.0 2.1 356520 44272 ? S Mar18 3:11 /usr/bin/php7.2 artisan horizon:supervisor site-prod-w-1-96p9:supervisor-1 redis --delay=0 --memory=128 --queue=default,ssl-manager --sleep=3 --timeout=60 --tries=3 --balance=simple --max-processes=10 --min-processes=1
forge 15357 0.0 2.1 356520 44532 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15358 0.0 2.1 356520 44504 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15359 0.0 2.1 356520 44196 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15360 0.0 2.1 356520 44340 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15361 0.0 2.1 356520 44568 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15362 0.0 2.1 356520 44388 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=ssl-manager --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15363 0.0 2.1 356520 44400 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=ssl-manager --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15364 0.0 2.1 356520 44236 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=ssl-manager --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15365 0.0 2.1 356520 44232 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=ssl-manager --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 15366 0.0 2.1 356520 44420 ? S Mar18 1:00 /usr/bin/php7.2 artisan horizon:work redis --delay=0 --memory=128 --queue=ssl-manager --sleep=3 --timeout=60 --tries=3 --supervisor=site-prod-w-1-96p9:supervisor-1
forge 20301 0.0 0.0 14856 980 pts/0 S+ 01:27 0:00 grep --color=auto artisan

Main Difference

But the main difference was that queues and supervisor were showing on production (not live yet really) in Horizon, not on staging. Why was that? They both had the same setups and should be identical as code wise they were. Both had Horizon up as well and only had one daemon up in Forge under daemons.

Horizon Configuration

When I checked config/horizon.php again I saw that we did have the ssl manager set up there as a supervisor and that we had two queues set up there:

'environments' => [
        'production' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default', 'ssl-manager'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 3,
            ],
        ],

        'local' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default', 'ssl-manager'],
                'balance' => 'simple',
                'processes' => 3,
                'tries' => 3,
            ],
        ],
    ],

Horizon was also installed as this file was added to config already on staging just as it was on production.

Horizon Run Redis Queues

So it seemed everything was run with Horizon on production, but not on staging. On staging we only had Horizon running with our own separate supervisor daemon under /etc/supervisor/conf.d/:

[program:laravel]
directory=/home/forge/staging.site.com/current/
command=php artisan queue:work --queue=ssl-manager --sleep=3 --tries=3 -vvv

process_name=%(program_name)s_%(process_num)02d
autostart=true
autorestart=true
user=forge
numprocs=1
redirect_stderr=true
stdout_logfile=/home/forge/.forge/laravel.log

So what we should have done is not start another supervisor manually for our own queues daemon nor do it via Forge. We just needed Horizon to get the supervisor and Redis queues up and running. Next question was, why did Horizon not load the configuration in the first place?

Horizon Checks

Horizon was up and running with our forge setup daemon so we did not need another

php artisan horizon

We double checked and no supervisor running under horizon

php artisan horizon:list
+----------------------+------+-------------+---------+
| Name                 | PID  | Supervisors | Status  |
+----------------------+------+-------------+---------+
| site-staging-5oFI    | 2610 | None        | running |
+----------------------+------+-------------+---------+

And yeah, none were being shown. Even when we restarted the server and deployed including the

php artisan horizon:terminate

Environment

In the end our main developer figured out the APP_ENV on staging was of course not like on production it being

APP_ENV=staging

, but that the config file was only loading for production. Once that was adjusted / we added:

    'environments' => [
        'staging' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default', 'ssl-manager'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 3,
            ],
        ],

all was fine again.