PHP offloading: Speeding up magento with Vala

  • Sharebar

PHP offloading: Speeding up magento with Vala

There is always an easy way to do things and there is also the less expected way. One of my favorite tricks to improve magento’s performance (any system really) has to do with preheating the content before it is even requested and keep it fresh by analyzing how often the contents of that page changes.

As always the main point is to offload the operation from one system to the other. In this case, since we are speaking about how to improve magento’s performance and speed up the site, especially for products and categories needles is to say that we are taking the data from the SQL server and sending it to cache. But what is it that we are caching and what are we using as our main storage?

MySQL as a cache storage

MySQL for what it is worth it’s awesome. Granted there are solutions out there that will definitely improve this system greatly, matter of fact there is a proposal to use Cassandra instead but for now to keep things simple we are using MySQL because the structure of our table is very simple (this is just an example right?) and we are going to keep the size of the tables at less than 1 million rows.

The principle

The principle is very simple: take an object that has been loaded and save it to cache, next time the same object is requested serve it from cache. In order to maitain the object fresh in cache operations like save, delete or update must delete the cached object.

Sounds poetic but that is wrong. Or a least I disagree with it now even more than ever. The real principle is more complex than that but let’s break it down to: take a deep look at your analytics and see how often the pages are being requested, how often your content is changing and put it all together into an algorithm to get prepare all of your content before it is even requested.

Why go through this trouble?

It all comes down to this: anticipating what pages your customers are going to be visiting next could be the difference between a happy customer and losing some good revenue. Why? Well there is never a guarantee that the page has been visited before and let alone that is in cached. There is also not guarantee that the content won’t be staled.

How do we do it?

In Magento there are several ways in which we can accomplish this feat but at the same time we have many ways to accomplish the same outside of magento.

So here we are again, trying to improve Magento’s performance by using Vala in our continue attempt to offload php to speed things up… wait say that again? Yeah, that’s right let’s speed up Magento, let’s make magento faster… just by using Vala.

So by now you know how awesome nginx is and you are using it for more than just serving web requests but also as a full page cache solution, right? If not you are truly missing out.

So while I cannot give you the full code ( I know I know…Sorry) I can show give you some snippets to get this baby rolling.

First define you schema (suggest you do this in a secondary database):

CREATE  TABLE IF NOT EXISTS `db`.`_vala_cache` (
  `cache_id` INT NOT NULL AUTO_INCREMENT ,
  `fpc_cache` TEXT NOT NULL ,
  `cache_hash` varchar(100) NOT NULL ,
  `customer_session` varchar(100) NOT NULL,
  `volatile_block_head` TEXT NOT NULL,  
  PRIMARY KEY (`cache_id`) ,
  INDEX `fpc_cache` (`fpc_cache` ASC) ,
  INDEX `cache_hash` (`cache_hash` ASC) 
)
ENGINE = InnoDB;

Now let’s see how we can connect to that database through Vala:

using Mysql;
public class Cache  {

    private string host = "localhost";
    private string user = "root";
    private string password = "pass4root";
    private string database = "vala";
    private int    port = 3306;
    private string socket = null;
    protected Database mysql = null;
    private ClientFlag cflag = 0;
    ~Cache(){

    }
    public void Cache () {

    }
    
    public int process_cache(string id){
        int rc = 0;
        //let's connect
        var isConnected = this.mysql_connect();
        if ( ! isConnected ) {
            rc = 1;
            stdout.printf(
                "ERROR %u: Connection failed: %s\n",
                this.mysql.errno(), 
                this.mysql.error()
            );

            return rc;
        }
        string sql = "SELECT * FROM _vala_cache where id="+ id +" LIMIT 1";
        rc = this.mysql.query(sql);
        if ( rc != 0 ) {

            stdout.printf("ERROR %u: Query failed: %s\n", this.mysql.errno(), this.mysql.error());
            return rc;
        }

        Result ResultSet = this.mysql.use_result();

        string[] MyRow;

        while ( (MyRow = ResultSet.fetch_row()) != null ) {

            stdout.printf("id: %s | data: %s\n", MyRow[0], MyRow[1]);
        }
        return rc;
    }
    public bool mysql_connect(){
      this.mysql = new Mysql.Database ();
      return this.mysql.real_connect(this.host, this.user, this.password,
                         this.database, this.port, this.socket, this.cflag);
    }
}
void main() {
    var cache = new Cache();
    cache.process_cache("1");

}

Save that file as mysql.vala
All good now what?

Let’s turn that piece of code into C:

valac -o mysql mysql.vala --pkg mysql -X -lmysqlclient -C

Now the resulting mysql.c can be compiled and linked to the fastcgi libraries as:

gcc -o mysql.cgi mysql.c -lfcgi

The resulting code can now be linked like this:

spawn-fcgi -a120.0.0.1 -p9000 -n ./mysql.cgi

Your nginx configuration won’t change much:

server {
        listen   80;
        server_name _;
 
       location ~ \.cgi$ {
                        # host and port to fastcgi server
                        root   /home/user/www;
                        index  index.html;
 
                        fastcgi_pass 127.0.0.1:9000;
       }
}

Of course you may be wondering how is this going to help… let me put it like this: Reading from the database using Vala is ridiculous fast and rendering code is crazy fast too.

Now keep in mind that the code in vala only reads from the database… ideally you want to have that compile code reading and writing to the database but also you want to add some code that will display the static cache page as well.

My initial tests give me a 200% speed boost and while that is awesome there is something you must know… My magento setup is quite complicated now. Now I have 2 database servers running in parallel and also a very complex algorithm to decide when to clean or refresh the cache and didn’t stand a chance against this siege:

siege -t15m -c300 -b -d -i magento.dev

Mainly because my computer resources weren’t enough. So until this code can withstand 300 concurrent users without destroying my computer then I will show all the code.

Now to speed things up just make sure you get all your current urls, request them periodically through vala and save them to cache. Keep in mind that in order to remove holepunching from the equation we need to implement a customer refresher where once a customer performs something that changes the header we refresh it in the database. That way while we assemble the pages we always get the right content for the current customer without having to holepunch it.

If you see any weird issues here please let me know.

VN:F [1.9.22_1171]
Rating: 10.0/10 (1 vote cast)
VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)
PHP offloading: Speeding up magento with Vala, 10.0 out of 10 based on 1 rating
 PHP offloading: Speeding up magento with Vala

About Luis Tineo

Husband, Father, performance improvement junkie, biker and video gamer, Linux user and in my day job I'm a Systems Architect at Blue Acorn.

Tags: , , ,

  • Tim

    Hmm what about in combination of setting the read DB in magento to this cached one in local.xml (sounds like fun) ..

    VA:F [1.9.22_1171]
    Rating: 0.0/5 (0 votes cast)
    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • http://www.kingletas.com letas

      Indeed. I actually have the code for reading the local.xml out of Magento and query the database based on Magento’s config but I thought it would be fun to show how to use Vala to optimize Magento without being concerned about the configurations. I will edit the post soon!

      VN:F [1.9.22_1171]
      Rating: 0.0/5 (0 votes cast)
      VN:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  • Pingback: The trustworthy bash to the rescue

  • Pingback: Caching is not performance optimization

  • Pingback: use sqlite3 with magento

  • Pingback: Update your SQLite3 database with Vala