Futureproofs API Optimisation

In an attempt to increase the speed of the Futureproofs API, we made several tweaks to the environment and the code, below is a summary of the ones I can remember.

Installed Profile Tools

I’ve used xdebug for many years, its comprehensive, but quite resource intensive. A few years ago facebook released xhprof, which is a PHP profiler designed to run on live systems (its uses very little resource). Unlike xdebug and cachegrind XHProf doesn’t run across all code automatically, you ‘start’ timing and ‘end’ timing by calling custom functions included in the PECL library. As I wasn’t entirely sure where we had issues I wanted to profile entire enpoints. This is where xhGUI came in useful.

xhGUI provides a very slick way to visualise the recorded data from XhProf. It copies the data into MongoDB and then provides funcationality such as comparison and searching. It also includes a PHP file that can be automatically appended/prepended (using .htaccess) to executed scripts so that we can profile its entire execution.

I found two v.useful articles when implementing this: https://blog.engineyard.com/2014/profiling-with-xhprof-xhgui-part-1 https://blog.engineyard.com/2014/profiling-with-xhprof-xhgui-part-2

Routing Updates

As we added extra functionality to the API we added more and more routes. Originally the routes were added by including the entire controller:

index.php

$app = new \Slim\Slim();
new \Futureproofs\Routing\Controllers\SomeEndpointController($app);
$app->run();

SomeEndpointController.php

class SomeEndpointController {
    public function addRoutes() {
        $app->get('/someendpoint', array($this, 'listEndpoint'));
        $app->options('/someendpoint', array($this, 'optionsEndpoint'));
    }

    public function listEndpoint() {
        //Return some JSON content here
    }
}

but by the time we had added 30 controllers, we realised this wasn’t going to scale. As an alternative, I created a Routes.php. This not only meant we only needed to load 1 file for all routes, but also meant I could reduce the number routes before the Slim router took over.

index.php

$app = new \Slim\Slim();

$app->hook("slim.before.router",function() use ($app){
    $routes = new Futureproofs\Routing\Routes($app);
    $pathInfo = $app->request()->getPathInfo();

    if (strpos($pathInfo, "/someendpoint") === 0) {
        $routes->addSomeEndpointRoutes();
    } else {
        $routes->addRoutes();
    }
});
$app->run();

\Futureproofs\Routing\Routes

class Routes {
    protected $app;

    public function addSomeEndpointRoutes() {
       $this->app->get('/someendpoint', '\Futureproofs\Routing\Controllers\SomeEndpointController:listEndpoint');
       $this->app->options('/someendpoint', \Futureproofs\Routing\Controllers\SomeEndpointController:optionsEndpoint');
    }

    public function addRoutes() 
    {
        //Other routes
    }
}

Removed Non PSR-0 Libraries

Having installed xhGUI I discovered that inclusing a library using the composer autoloader that wasn’t PSR-0 compliant was slowing down every request to the API. I won’t name and shame the forementioned library, but by forking it on Github, making it PSR-0 compliant and importing the forked code into my library actually sped up the API by nearly 5ms for each request.

Used a more efficient Monolog logger

We use monologger within the API to log errors, but also to log information which may be useful (such as unauthenticated requests). What I’d not realised however is that the RotatingFileHandler isn’t very performant. Each time its called, it scans the file system to check for outdated logs and then removes/renames them as needed.

It turns out there is a process that can be installed on Nix based systems which replaces this functionality but is a lot more efficient called Logrotate. By having it remove old files monolog can just concern itself with writing to a text file (known as the StreamHandler).

Removed unused Apache modules

This one was a bit of ‘micro-optimisation’… By removing unused Apache modules, each request requires less resources. I removed each of the modules below, testing the API as I went. ` sudo a2dismod authn_file sudo a2dismod authz_default sudo a2dismod authz_groupfile sudo a2dismod proxy sudo a2dismod proxy_http sudo a2dismod cgi sudo a2dismod autoindex sudo a2dismod setenvif

Removed .htaccess checks

All of our apache configuration for the API is in the virtual host config. This means we have no need for the .htaccess file to be read on each request. Adding ‘AllowOverride None’ to the virtual host config prevents this read:

<Directory /var/www/>
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

Turned on GZipped JSON responses

This simple update is likely to provide the most significant performance increase. All that was required was to add this line to the virtual host configuration:

`AddOutputFilterByType DEFLATE application/json`

Optimised the code

I’d already made some code improvements (as mentioned above) however I also reduced the checks performed when browsers make an OPTIONS request and removed the returned array (the browser is only interested in the header, not the body).

Auth System Optimisation

All the above changes were also added to the auth sytem (this is called on almost every API request).

To Do In Future

  • Upgrade to PHP5.6
  • Remove password_verify substitute
  • Remove unused kissmetrics library & consumer
  • Add caching layer (redis)

Ben Squire

Read more posts by this author.

Leicestershire, United Kingdom https://squired.co.uk