Add context on your Blackfire timeline from a Google Alerts RSS Feed

Observing your application is essential to be able to find what needs (and what does not need) to be optimized. Understand your traffic spike is key but when an external source lead to those traffic spikes, that’s not any easy path to find out the exact reason.

As an example, your CEO talks about your brand on a TV show, this automatically leads to a raise of your traffic few seconds later and by ricochet, potential performance issues: that’s what I call in this article the TV show effect. Other names can be found for these external events leading to traffic spikes: Slashdot effects or Hug of death. Of course, most of the requests on your website would be absorbed by your reverse proxy in front (Varnish, Faslty,…), but some of those requests won’t and would lead to an increase of the resource consumption.

Blackfire is a great Application Performance Monitoring (APM) tool to observe and gather data about consumed server resources like memory, CPU time, and I/O operations. On the Monitoring section of your Blackfire dashboard, the timeline helps you to analyse the behavior of your application, by giving a lot of metrics in a specific timeframe of your traffic. Blackfire also provides a great (and not so well known) feature to spot markers on this timeline. These markers are a way to add more context for your observers to better understand what’s going on (when and why?).

We saw in a previous tutorial How to automatically spot markers on your Blackfire timeline for any occuring Upsun infrastructure processes. In this current tutorial, I will showcase how you can import any new Google alerts that pops up and spot markers on your Blackfire timeline. As a summary for this tutorial, we will import, via a cron, any new RSS Feed entries in a Symfony database (SQLite), and spot corresponding markers on your Blackfire timeline.

Google alerts and Blackfire markers

Create a new Google alert RSS Feed

The first step is to create a new RSS Feed alert.

To do so, connect to your Google account and go on Google alerts page and follow this official tutorial. Please fill it with the following options:

Create a fork of the blackfire-google-alert-example repository

To ease your work, I already developed a small Symfony application that contains the minimum codebase for importing all feeds from a Google Alert RSS Feed in your database and spotting corresponding markers on your Blackfire timeline, using a cron.

You can start by creating a fork of this blackfire-google-alert-example GitHub repository in your own GitHub organization for later usage.

🚨 Please note: for this tutorial, we use a Symfony application but feel free to reuse the business logic and come up with a solution using any other framework you like.

The source code in this GitHub repository is already Upsun-ready. After creating a fork, you can clone it locally and create an empty Upsun project. Use the 3 following command lines from the root of your source code, and follow prompts:

git clone git@github.com:<OWNER/REPOSITORY>
cd <REPOSITORY>
upsun project:create

Final output should be:

  ...
  The Upsun Bot is activating your project

      ▄     ▄  
      ▄█▄▄▄█▄  
    ▄██▄███▄██▄
    █ █▀▀▀▀▀█ █
       ▀▀ ▀▀   

  The project is now ready!
  <PROJECT_ID>

• <OWNER/REPOSITORY> is your forked <REPOSITORY> in GitHub.
• <PROJECT_ID>: keep this project ID in mind for later use

At this step, your Upsun project is not deployed yet.

We need first to pass to the application your previously created RSS Feed uri via an environment variable.

Define your RSS Feed uri in an environment variable

Before deploying, please note that this blackfire-google-alerts app need you to define an environment variable for your RSS Feed uri (<GOOGLE_RSS_ALERT_URI>).

For local usage of this application, you can define this GOOGLE_RSS_ALERT environment variable in a .env file:

GOOGLE_RSS_ALERT=<GOOGLE_RSS_ALERT_URI>

As Upsun projects does not import .env file (local Symfony server), there are 3 ways to define environment variables for your Upsun environment/project.
You can either:

• use an .environment file

  GOOGLE_RSS_ALERT=<GOOGLE_RSS_ALERT_URI>

  Then you need to add this file to your Git history

  git add .environment && git commit -m "adding GOOGLE_RSS_ALERT envVar"

• use the Upsun CLI.

upsun variable:create --level project --name GOOGLE_RSS_ALERT --prefix env --value <GOOGLE_RSS_ALERT_URI>

• define it from the Upsun Console: to do so, please follow steps described in the corresponding documentation page (In the Console tab)

Add a source integration to your fork

Then, you need to add a source integration to sync your Upsun project with your new forked GitHub repository.

upsun integration:add --type github --repository <OWNER/REPOSITORY> --token <GITHUB_ACCESS_TOKEN> [--project <PROJECT_ID>] 

• <PROJECT_ID> is the ID of your new blackfire-google-alert project ID (from upsun project:create command).
• <GITHUB_ACCESS_TOKEN> is your generated GitHub API Token.

For more information about source integration, please refer to the enable source integration documentation page.

Blackfire Timeline testing

From the Upsun Console of your project, wait for the first deploy to be completed (~3 minutes) and then click on Launch Blackfire button on the left menu.

🚨 Please note: If you would like to automatically open your current project Console dashboard from your terminal, you can use this command from the root of your source code:

upsun web

You then land on the Blackfire dashboard (Health report). CLick on the Monitoring menu item on the left. Markers should appear on the timeline, such as:

🚨 Please note: The blackfire-google-alerts app imports RSS Feed entries via a cron that runs every 5 minutes, so depending on the timing, you probably need to wait a bit for the first import to take place.
If you want to manually trigger the first import, you can also execute this command:

upsun ssh 'php bin/console blackfire:import-google-alerts'

Conclusion

Adding more context to your Observability timeline is crucial if you are running applications with a lot of traffic and if you want to be able to detect what i called TV show effects.

In this guide, we took a look at how to automatically add context to your Blackfire timeline using an external source (here, Google Alert RSS Feed). Some next steps could include importing any other external source of information that could explain traffic spikes on your application, or give a way for your marketing team to manually spot incoming events on your Blackfire timeline?

• In 2 weeks, your CEO is willing to be at a TV show, talking about your brand?
• A new 1 month nurturing campaign is planned in 3 weeks (start|end markers?)?

Possibilities are infinite, and all of these features to add context on your Blackfire timeline would help a lot your observers/developers to analyse your traffic and understand why and when a performance bottleneck is happening.

I keep you posted, I will soon write another episode to go further on contextualisation of your observability strategy.

Remain up-to-date on the latest from us over on our social media and community channels: DEV.to, Reddit, and Discord.