Imagine you push to your project's repository and the documentation updates automatically on your website. Magic? No, Webhooks!
Do your homework - write docs!
If you have a lot of different products like we do, it's important to provide detailed documentation for users. It may be even more important to keep the documentation up to date - and that can become a hassle the more products you have.
That's why we implemented an automatic process to keep our documentation organized as well as up to date and centralized on our website, matching with our UI and style. You can check out all of our documentation on the website.
Every product has a docs folder in its repository which is automatically transmitted to our platform and converted to blade templates.
Communication is magic
As soon as we push some code to an app repository, for example Expose, the repository notifies our platform. The Laravel app then executes a command and the Expose documentation is automatically updated from the Expose repository right to our website. Sounds like magic, right? 🦄
The secret sauce behind all this sorcery is Webhooks. Webhooks are, simply put, a great way to communicate from server to server. We can use them to integrate payment providers, automate processes based on what's happening and much more.
How do Webhooks work?
In short, Webhooks are HTTP callbacks. Server 2 has an endpoint that can be called if something happens. Unless like with regular polling, Server 2 doesn't ask Server 1 permanently if something happened, but vice-versa: Server 1 is doing its tasks, and when something happens that is relevant for Server 2 (e.g. a payment is processed), it sends a request with payload data to the endpoint that Server 1 provides. If you want to learn more, you can check out this page of the GitHub documentation.
I made a fancy little diagram for our example to visualize what is happening between GitHub and our Laravel App:
Add a Webhook to your GitHub Repository
Adding a Webhook to your GitHub Repository is easy: Go to »Settings → Webhooks« to add a new one with the following settings:
You'll have to adjust the Payload URL with for your application and change the secret. After saving this, GitHub will send a test payload to your application - this will fail for now, because we haven't done anything on the Laravel side yet. But we will change this right now!
Testing Webhooks during development
GitHub needs a Payload URL for the Webhook setup, so working with a localhost development URL would not work. But you can't deploy your code to production every time you make a little change, right? For this reason, we developed Expose – set up a free account in seconds, share your local site and receive the payload of the Webhook on your local machine. Inspect and replay the request until you are ready to deploy to production.
Handle the Webhook in Laravel
We have to set up an endpoint in our api.php routes file like this:
Route::post('/update-docs', [\App\Http\Controllers\GitHubWebhookController::class ,'handleDocsHook']);
GitHub is sending a post request, so we need a post route that calls the GitHubWebhookController. This is the entry point to our implementation and contains the method handleDocHook() with a Request parameter:
<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use App\Jobs\UpdateDocumentationJob;
use Illuminate\Validation\UnauthorizedException;
use Symfony\Component\HttpKernel\Exception\HttpException;
class GitHubWebhookController extends Controller
{
/**
* @param Request $request
*/
public function handleDocsHook(Request $request)
{
$this->validateGithubWebhook(config('app.docs.github_webhook_secret'), $request);
$repository = $this->detectRepositoryName($request->get("repository"));
dispatch(new UpdateDocumentationJob($repository));
}
}
In this case we just want to know that a push has happened in a specific repository, we don't care about the content of the data sent by GitHub – except for the repository name and the signature for the authentication. The latter ensures that the request to our Laravel application is valid. We do this by calling the validateGithubWebhook() method and check the signature in there – if the signature is invalid or false, it throws a BadRequest or Unauthorized HttpException. Remember to set your GitHub webhook secret in your .env file and add the entry to your config (config('app.docs.webhook_secret') in this case).
# GitHubWebHookController
/**
* @param string $token
* @param Request $request
*
* @return void
* @throws HttpException
*/
private function validateGithubWebhook(string $token, Request $request)
{
if (($signature = $request->headers->get('X-Hub-Signature')) == null) {
abort(400, 'Signature header is not set.');
}
$signatureData = explode('=', $signature);
if (count($signatureData) !== 2) {
abort(400, 'Signature format is invalid.');
}
$webhookSignature = hash_hmac('sha1', $request->getContent(), $token);
if (!hash_equals($webhookSignature, $signatureData[1])) {
abort(401, 'Could not verify request signature ' . $signatureData[1]);
}
}
The method detectRepositoryName() extracts the name of the repository from the request, for example beyondcode/tinkerwell. We need the full qualifier to receive the documentation of the repository in the next step, which is done in the UpdateDocumentationJob.
# GitHubWebHookController
/**
* @param array $repository
*
* @return string
* @throws HttpException
*/
private function detectRepositoryName(array $repository): string
{
if (array_key_exists("full_name", $repository)) {
return $repository["full_name"];
}
abort(400, 'Invalid payload');
}
It can take some time to update the documentation, especially when you have a lot of files to process. Therefore we want the process to be executed in the queue and dispatch the UpdateDocumentationJob in handleDocsHook().
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Artisan;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use App\Console\Commands\UpdateDocumentationFromGitHub;
class UpdateDocumentationJob implements ShouldQueue
{
use Dispatchable, Queueable;
private $repository;
/**
* Create a new job instance.
*
* @param string $repository
* @return void
*/
public function __construct(string $repository)
{
$this->repository = $repository;
}
public function handle()
{
Artisan::call(UpdateDocumentationFromGitHub::class, ["repository" => $this->repository]);
}
}
Create an artisan command
The actual processing takes place in an artisan command that is called inside the job to keep it short and clean. You can create an artisan command by calling
php artisan make:command UpdateDocumentationFromGitHub
This makes the testing during development easier, because you don't have to fire a webhook every time.
This command handles a few things – I'll go over some helpful details instead of just pasting the whole code here, because the process will be different for each application.
Pull the documentation from the GitHub repository
There are a few steps to take for this command, so I highly recommend that you sum them up in private methods and call these from the handle() method of the command. This keeps your code nice and tidy and gives you the possibility to log execution of every step like this:
We have registered our command with the signature docs:update and it takes the full repository qualifier (beyondcode/expose in the screenshot above) as an argument.
For a start, the command clones the whole repository from GitHub. Why don't just use the ZIP file GitHub is providing, you ask? Depending on your .gitattributes configuration, it won't contain the docs folder. By cloning the repo via SSH, we make sure we get everything the repository contains.
// clone the GitHub repository into the temporary directory
$process = new Process(
[
'git',
'clone',
'[email protected]:' . $this->argument('repository') . '.git'
],
$this->commandStorage);
$process->run();
We use the symfony/process package to execute CLI-level commands in our application. The first argument of the Process() constructor is the command itself, the second (optional) defines the working directory where the git clone is performed. $this→commandStorage contains the path of a temporary directory created in the constructor of the Artisan command, located in the storage_path of the app:
$this->commandFolder = "/tmp/" . uniqid("tmp_");
$this->commandStorage = storage_path("app" . $this->commandFolder);
File::makeDirectory($this->commandStorage);
Please note that a package like spatie/temporary-directory won't work in this case - Process() does not seem to have access to these.
Detecting the docs folder
After cloning the repository, we should check if it even has a /docs folder!
$assumedDocsFolder = $repositoryFolder . "/docs";
$repositoryDirectories = collect(Storage::directories($repositoryFolder));
if (!$repositoryDirectories->contains(Str::substr($assumedDocsFolder, 1))) {
throw new \Exception("docs folder not detected");
}
return storage_path("app" . $assumedDocsFolder);
With the handy function Storage::directories() we can list all directories contained in a given path. It searches for the docs/ folder and returns the full path for further processing if it exists.
Moving the docs to your application
The last step is to copy the docs directory to the desired spot in your Laravel application by using the path we just detected. In our case, we duplicate the docs to /resources/views/docs.
Due to more great Laravel helpers, resource_path() and File::copyDirectory(), this is just two more lines, so we execute this right in the handle() method of the command:
$copyDocsTo = resource_path('views/docs/' . $this->getDocsFolderName());
$this->comment('đź“‘ Moving documents into resource storage ' . $copyDocsTo);
File::copyDirectory($localRepositoryDocsFolder, $copyDocsTo);
What to do with all the markdown?
Now you have a docs folder in the resources of your Laravel app that is automatically updated whenever you push to your projects repository. To integrate it properly into your website, you could use a markdown parser like Parsedown and give your docs a nice layout. This step of the process is too individual to describe in a single tutorial, but you'll find a way for sure!
