New Implementation - Cache System

Problem

Using flat text file as storage instead of proper SQL database, we have two main problems usually: data security, and data extraction efficiency. In this update, I mainly focus on the latter.

As a performance test, I generated 500 test posts automatically, and the result was BAD. The Pages took a very long time to load. The main reason for that was all the plugins and index page tend to load all the posts files into memory and process them in every page request. Reading a big file into memory isn't a problem, but when there are a large number of files, even small files, it just takes forever to open them.

Solution

To Improve it, I would have to reduce the number of files that I need to open per page request. And the answer could be just one, 1 file VS all the post files. - The Cache File.

I can cache the data that I will need for the plugins beforehand and save it in a file. For example, if I need to display category on the sidebar, I will need the category name, and the post count of that category. To cache this information, I can simply create a plugin function that goes through all the posts, counts the number of posts of all the categories and save the result in a file. When the category plugin is triggered, it reads that cache file and displays the information, instead of going through all the posts again!

To make this more useful, we can also cache some details of the posts that are in this category, for example the post ID and the title, which are what we need in order to display an Archive view.

The same applies to other plugins such as tag cloud and archive.

Drawback

One known side effect of this new system is, the cache system doesn't pick up new posts, admin needs to update the cache files manually. Although I could set it to update automatically when a new post is published, or by a cron job, I might leave to be discussed or think about later.