Organize Realtime Job Management With Laravel and Vue Step by Step [Part 1]

Introduction

The Laravel ecosystem contains perfect instruments for creating background jobs, handle jobs with queues, scale and monitor queues with laravel/horizon. It’s work well out of the box, with minimal configuration, and seems enough for a most internal cases like sending email and notification, generation thumbnails for image, creation pdf/excel reports. But in some cases we need more feedback from background job execution process, such as ability to see execution logs per each job, got real-time notifications about start/finish job, even ability to see the progress of job execution in real-time.

TL;DR

You can find demo app here https://github.com/Insolita/InteractiveJobs

Preparation

Before start we must have runnable laravel application with configured database connection, redis connection, prepared user authorization and seeded with at least 2–3 users accounts.

also we needs install laravel/horizon

In .env file we should have `QUEUE_DRIVER=redis`

So, now we create first job php artisan make:job DummyJob and simple service which will imitate external service with complex logic

Also the controller for triggering to run our job

Route::get(
'/dummyJob', 'JobController@dummyJob'
)
->name('dummyJob')
->middleware(
   'role:admin'
);

Run php artisan horizon in terminal and touch http://your.app/dummyJob. Ensure that job was executed.

Saving job logs

The essence of the below actions is to make information about executing job available for LogHandler

• Create new migration

php artisan make:migration create_job_logs --create job_logs

Schema::create('job_logs', function (Blueprint $table) {
     $table->increments('id');
     $table->nullableMorphs('loggable');
     $table->string('level');
     $table->text('message');
     $table->text('context'); //Or json/jsonb if your db supported
     $table->text('extra');   //Or json/jsonb if your db supported
     $table->dateTime('created_at');
 });

We use polymorphic fields for maximize flexibility

• Add new config file jobs.php

<?php
return [
    'context'=>null
];

• Create Database Log Handler for Monolog and Factory for integration. If this step is not clear, see https://laravel.com/docs/5.6/logging#creating-channels-via-factories

• Create ServiceProvider for our module, and register listeners for queue events. Don’t forget to add in config/app.php

When Job Processing event triggered, we define jobs.context from payload data, and when job will be finished or failed, jobs.context became nulled

Restart horizon, and run http://your.app/dummyJob 5–6 times. Now you can see in database job_logs that each record saved with job identity.

Honestly, laravel/horizon not necessary for this case. The main different is that the horizon provide integer increment job ids, but simple queue generate string job identifiers.

But wait! What if we don’t want to save logs for each job?

There are some ways for solve it: write logs for custom Job classes, for some queues, or by job tags

app(QueueManager::class)->before(function (JobProcessing $event) {
   $payload = new JobPayload($event->job->getRawBody());
   //if(in_array($payload->commandName(), [...allowed commands])){
   //if($event->job->getQueue() == 'loggable'){
   if(in_array($payload->tags(), ['loggable'])){
      Config::set('jobs.context', ...});
   }
}

Make Model and Controller for Job Logs

They contain absolutely nothing extraordinary

Views

(i use default laravel layout with bootstrap4 preset)

So now our app looks like this

/logs

The main disadvantage of this stuff is that it related to Job actions, but not to business logic. And we have several ways for improve it.

Firstly, if we want to make relation with model, passed in Job constructor we can choose convention way and define it as displayName property

Make replacement in JobController

//DummyJob::dispatch(...)->onQueue('default');
UserDummyJob::dispatch(Auth::user())->onQueue('default');

And touch http://your.app/dummyJob several times from different user accounts. All log records should be groupped by users. It give us ability to create relation in User model

public function logs()
{
    return $this->morphMany(JobLog::class, 'loggable');
}

Other way — is make own base class for Jobs, that will provide more abilities

Seems, we need to create Job Model

Schema::create('jobs', function (Blueprint $table) {
    $table->increments('id');
    $table->string('queue')->default('default');
    $table->text('payload')->nullable();
    $table->text('report')->nullable();
    $table->string('state');
    $table->integer('progress')->nullable();
    $table->string('command');
    $table->smallInteger('attempts')->unsigned()->default(0);
    $table->integer('created_by')->unsigned()->nullable();
    $table->dateTime('created_at');
    $table->dateTime('finished_at')->nullable();
});

Also we make separated JobState class for present Job state. You may use it as mutator for model, if you want. I leave it in separated method jobState()

So, if you don’t have a headache, give me your claps, and go to the next part->