Restore Ubuntu to the default settings without re-installing it

As a web developer (read app freak) I am always installing new IDEs, media players and libraries to try out. This madness cycle can definitely take a toll on your computer and slow it down to the point of no return.

Usually that is when you'd decide to format the computer and start anew. So when my wireless network was experiencing some hiccups and my computer will crash every 2 seconds for no reason or worse when it started using all of the swap memory I knew it was time to wipe it clean. Then I look at my home folder and I have over 300GB of data and I am kind of lazy. That is when I think to myself, no problem the home folder is in its own partition, right? Nope. It is not *BUMMER*

I don't want to re-install and reconfigure all of the programs I use but don't want to restart my computer every hour or so - that is when I remembered that some bash commands could save my day.

Currently I am using ubuntu 12.04 and I want to restore it to be the same it was when I first installed it, sure no problem let's start by getting two simple files from the ISO file (you still have the iso file right?):


casper/filesystem.manifest
casper/filesystem.manifest-remove

or just casper/filesystem.manifest-desktop for *buntu 11.10 or earlier
Now let's combine these 2 files:

comm -3 <(cat filesystem.manifest | awk '{print $1}' | sort) <(cat filesystem.manifest-remove | sort) > default.txt

If you have Ubuntu 11.10 or earlier then to get the default.txt file do:

cat filesystem.manifest-desktop | awk '{print $1}' | sort > default.txt

default.txt contains a sorted list of all the programs our default installation should include. Now let's get the software currently installed in our computer:

dpkg --get-selections | awk '{print $1}' | sort > currently_installed.txt

If we diff the default software vs what is currently installed, we can get what was added:

 diff -u default.txt currently_installed.txt | grep "^+[^+]" | cut -c 2- > additions.txt

or what was removed:

diff -u default.txt currently_installed.txt | grep "^-[^-]" | cut -c 2- > removed.txt

Now if we want to remove all of the additions, including the configuration files:

sudo apt-get purge $(cat additions.txt)

To add back the files we removed:

sudo apt-get install $(cat removed.txt)

Remove the configuration files:

rm -rf .gnome .gnome2 .gconf .gconfd .metacity

That should bring your current ubuntu based to the default state (I believe it should work with every Linux Distro, but the judge is still out there).

This is a bit radical though, before removing any files make sure that you only remove anything that you don't want to keep. And just in case, make a backup of your files just in case.

Varnish implementation hints and good to know

Sharing knowledge is always a good idea. Lately I have been doing lots of integration with Varnish and Magento with different clients and it will be interesting to find out if you have any ideas or suggestions (read improvements) to make it better and ultimately help the community even more:

Setting up varnish in switch mode:

In my home directory I always have a bash aliases (vim ~/.bash_aliases) file where I add the following iptable rules:

alias ipforward='sysctl net.ipv4.ip_forward=1'
alias varnishon='iptables -t nat  -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8888'
alias varnishmasq='iptables -t nat -A POSTROUTING -j MASQUERADE'
alias varnishoff='iptables -t nat -D PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8888'
alias varnishstatus='iptables -L -t nat | grep -q 8888; if [ "test$?" = "test0" ]; then echo "Varnish On"; else echo "Varnish Off"; fi'

This allows me to switch varnish on and off without changing the existing architecture. The best thing about this is that if something were to go wrong I can immediately turn varnish off and test the web server alone. Also internal routes won't go through varnish which let's me test side by side the pages and speeds.

Setting up a varnish on flag:

When Implementing ESI, always set a variable to tell you whether varnish is on or not. For instance on my varnish configuration file (/etc/varnish/default.vcl), at the very end I have:

   
/**
   * Set a flag for web store to check if varnish is on or not
   * Very important when doing switch on / off installations with ESI  
  **/

 set req.http.X-Varnish-On = 1;

    if (isset($_SERVER['HTTP_X_VARNISH_ON'])): 
           
    else: 
           getNormalContent() ?>
   endif;

I have it a the very because, all of the whitelist and passes have already been checked – so I am assure that the only pages left are “varnished” pages.

Remove all the cookies

The only cookie value Magento requires is the frontend cookie:

//check that the cookie is still set
if (req.http.Cookie){

 set req.http.X-Cookie = ";" +req.http.Cookie;
 set req.http.X-Cookie = regsuball(req.http.X-Cookie, "; +", ";");
 set req.http.X-Cookie = regsuball(req.http.X-Cookie, ";(frontend)=", "; \1=");
 set req.http.X-Cookie = regsuball(req.http.X-Cookie, ";[^ ][^;]*", "");
 set req.http.X-Cookie = regsuball(req.http.X-Cookie, "^[; ]+|[; ]+$", "");
}

// Remove cookies
unset req.http.Cookie;

Then assuming you only have the x-cookie set for the frontend, in your php before the headers are sent:

$_frontend = isset ($_SERVER['HTTP_X_COOKIE']) ? $_SERVER['HTTP_X_COOKIE'] : null;
if ($_frontend){
    list ($name,$value) = explode("=",$_frontend);
    $_COOKIE[$name] = $value; //setCookie didn't work for me here
}

Clear Cms / Product / Category pages

Update the content frequently? When you save a cms page, product or category page simply hear the corresponding event and “purge” or “refresh” that URL. The important thing to remember is that if you can: 1-) use cURL in BASH over PHP's cURL and 2-) when clearing the product URLs you also need to clean the categories as well. In the terminal you could do:

curl -X PURGE URL_TO_CLEAN

Clear / Refresh all pages

A favorite of mine – when finding all of the URLs in the system, use Magento's sitemap resource collections:

    //get all the url rewrites urls and cms pages (including the home page)
    public function getAllUrls() {

        $collections = array('sitemap/catalog_category', 'sitemap/catalog_product', 'sitemap/cms_page');

        $baseUrl = Mage::app()->getStore()->getBaseUrl();
        $storeId = Mage::app()->getStore()->getId();
        $urls = array();

        foreach ($collections as $collection) {
            $collection = Mage::getResourceModel($collection)->getCollection($storeId);
            $key = md5(serialize($collection));
            if ($data = $this->getDataFromCache($key)) {
                $urls = array_merge($urls, $data);
            } else {
                foreach ($collection as $url) {
                    $urls[] = htmlspecialchars($baseUrl . $url->getUrl());
                }
                $this->setKey($key)
                        ->setData($urls)
                        ->saveDataInCache();
            }
            unset($collection);
        }

        $urls = array_merge($urls, $this->getSearchUrls());
        //account for the homepage 
        $urls[] = $baseUrl;

        return $urls;
    }
     public function getSearchUrls() {
        $urls = array();
        $collection = Mage::getResourceModel('catalogsearch/query_collection')
                ->setPopularQueryFilter(Mage::app()->getStore()->getId())
                ->setOrder('popularity', 'DESC');

        $catalogSearchHelper = Mage::helper('catalogsearch');
        foreach ($collection as $search) {
            $urls[] = $catalogSearchHelper->getResultUrl($search->getData('name'));
        }

        return $urls;
    }

Please note that I use some extra functions here: setKey, setData, saveDataInCache & getDataFromCache. Ultimately is all about performance and I rely on Magento's cache to save the collection of urls. That will get you all of Magento URLs that your system might be caching. Now you could iterate that array and either refresh or purge varnish's cache