Converting caffeine to code, shooting pics, and cruising the roads of life.

Archive for the ‘Tips and Tricks’ Category

Free Template

Tuesday, May 16th, 2006

Over the past few days, I have been working on my new website pretty much non-stop. So today, I took a break and cleaned up some code in a template I made a few months ago. This template was what I used to run on this website’s old server, devnulled. Since I only use devnulled for private stuff now, I am going to released the template. So without anymore delays, I present it to you.

DevNulled Template
Download: http://www.codymays.net/files/devnulled_hosting_template.zip
Demo: Here

I included as many of the PSD files as I could find and optimized the images a bit. Hope you guys like it.

Bring back that old computer! Make it useful for modern day activities, but with a twist!

Thursday, May 4th, 2006

So you have an old computer stuffed in your closet, but you think it’s too old to do anything? Probably not, but do you remember the good old days of 80×25 text mode? Many people think that this is completely useless nowadays, after being spoiled with desktop environments like KDE, gnome, and Aqua. The truth is that they’re wrong. You can do many things in text mode that you can do in vga/vesa.

First off let me tell you about how we’ll get this thing started. I am assuming you can at least turn the computer on and get into the BIOS. You’ll need a CD drive, a harddrive of at least 4gb, and an internet connection on the old computer. I’ll be using FreeBSD in this article, but many of these applications are easily available in source for Linux or any other Unix.

Install FreeBSD

First off, you’ll need to grab the latest FreeBSD iso from FreeBSD.org and burn it to a cd. Now, go into your computer’s bios and set the cd drive to be the first boot device, stick the cd in, and reboot. Once the cd starts up, you should see the following:
sysinstall

Now, navigate down to Standard and press enter. Press enter on the next menu that comes up telling you that you need to setup a partition scheme and you should see the screen below:

Fdisk

This is where we create what is known to BSD users as a slice. A slice is basically one of the four “primary” partitions in DOS terms. I’m assuming these computers have small harddrives, so let’s just go ahead and use Auto, so press the ‘A’ key. Now the list should have the slice added. Press ‘Q’ to save the changes and move to the next step.

A menu should come up with three options at the bottom, they are BootMgr, Standard, and None. Move down to Standard, press enter, and press enter again on the Ok in the next menu.

FreeBSD Boot manager menu

Now you should see a screen very similar to what you saw when editing the slices. I’m not going to go into details about slices or partitions here, but a quick look in the FreeBSD Handbook should give you plenty of information. Just go ahead and press ‘A’ on this window, then press ‘Q’.

disklabel

Shortly after you press Q, you should get a window asking you to choose your distribution. Scroll down with the arrow key and press enter on developer. Immediately after pressing enter, a window will come up asking if you would like to install the FreeBSD ports collection. Choose yes. Now press the arrow key back up to X Exit and press enter.

Choosing a distrobution

Enable Ports

A menu asking you to select the media type should come up now. Since we are using the CD, select CD/DVD and press enter. Press enter on the screen that tells you this is your last chance to backup and FreeBSD will begin to install.

Choosing a media

After a few minutes you will see your success. :) Press enter.

Success

You should now see a menu asking if you want to configure any Ethernet or SLIP/PPP network devices. I will assume you have one and let’s walk through it. Press enter with “Yes” highlighted.

Choosing a distribution

After pressing enter, you will see a menu with a list of network devices. You should select the device from the list and press enter. Normally your NIC is the first in the list, but this may be completely different on your setup.

Network device list

Select No on the IPv6 configuration window that comes up and Yes on the DHCP menu. Enter a hostname in the input box. The hostname can be anything you wish, but try to keep it short. Press enter until “Ok” is highlighted and press it again.

A window should now pop up asking if you want to configure the machine as a network gateway. Choose no here. After that, you should see a window asking if you would like to configure inetd and the services it provides. Choose “Yes” here, because we are going to want to enable ftp access to the machine. Now, the usual enabling-network-services-can-provide-a-security-hole menu comes up. Go ahead and choose “Yes” again. Press “Yes” one more time to bring up an editor.

EE the editor

Remove the #’s from in front of both ftp’s so that it looks like this:

Editing inetd

Press Escape and a menu will open and press the A key.

Now we get into quite a few configuration menus. First, a menu asking if you want to enable SSH will come up, enable this. SSH will allow us to have a headless system so you don’t need a monitor for the old dinosaur, but only an SSH client. Do NOT enable anonymous ftp access to the box, that’s a very stupid thing to do. Select no for the next two menus having to do with NFS setup. You may edit your console settings with the menu that comes up yet, but I didn’t bother to. Of course you’ll want to set your time zone, so do that when the menu comes up, just make sure you choose no on CMOS clock menu.

We’re almost done! Enable the Linux binary compatibility when the menu comes up. You can enable the mouse in the menu that comes up next, but it’s not required. Don’t bother browsing the packages because we will be installing ports.

Linux compatibility

You must choose yes on the window that asks if you would like to enable any additional user accounts. This will be essential for when we go to setup SSH because I’m not going to enable root SSH, but allow root access over SSH via sudo. After selecting yes, you will see a window like the one shown below. You will want to select Users, then you will see the second image below. You only need to add a username and password. Once you have done that, change the shell text box so that it says /bin/tcsh. The 3rd image below should be what you have before you select okay. When the original menu comes back up choose Exit.

group editing

User edit

User edit 2

Go ahead and follow the directions for setting the root password that will come up next. Make sure you make this password high security because if anyone gets root access to your machine, you’ll get to start all over. :) Choose no on the next menu to visit the configuration menu again. Now chose “Exit Install” and follow the directions.

Exit sysinstall

Configuring FreeBSD

After the install is done you will be directed to reboot your machine, after taking the cd out. When you boot it up you will see a message that says to just type a bunch of junk and press enter, do so. This is for unblocking entropy and will only happen on this boot. Once it is finished booting up, you should see this:

first light

At the login prompt, type root as the username and then the password you set for root. If you don’t already know, root is the most powerful account available in a Unix system. Root allows you to do anything you want, including destroy the system. This is why you should not enable root over SSH or give out your root password. We’ll be using root here to install ports and setup some configuration files.

The first thing we should do is update our ports collection so we have all of the latest bug fixes. So at the prompt type the following:

hostname# pkg_add -r cvsup-without-gui && rehash

pkg_add allows us to install packages without having to compile them. I prefer to compile my ports, but you are more than welcome to use pkg_add. Rehash causes the internal hash table of the contents of the directories in the path variable to be recomputed, with the new binaries we just installed.

Here comes the fun part that can take a while, depending on how out dated your ports collection is. Run the following:

hostname# cd /usr/share/examples/cvsup
hostname# ee ports-supfile

You should now see a screen like the one below:
[IMG eeportssupfile]
Scroll down until you find this line:

*default host=CHANGE_THIS.FreeBSD.org

and change it so that it’s one of the domains listed here or just use the following:

*default host=cvsup5.FreeBSD.org

Now press ^[ (the escape key) and then press A to save our changes. What this does is checkout the latest ports collection from one of the cvs servers (cvsup5 in this case). So to start it, do the following:

hostname# cvsup ports-supfile

After a few minutes this should finish and take you back to a command prompt.

Installing Sudo

I'm going to make you install sudo as I said above. Once this is installed, I'll show you how to setup a "headless" system so you can unplug the keyboard/monitor and push the machine back into the closet. You'll just need a network and power cable. Let's get started! Sudo stands for superuser do. This allows any user added to the sudoers file to execute certain commands, all commands, or even get a root shell without having to directly login as root. This has quite a few benefits. The main thing is that you don't have to enable root over SSH and it makes you think of what you are about to execute as root. This can keep you from screwing your system over, so from here on out, let's consider it a bad thing to use the full root user unless absolutely needed.

First thing we'll need to do is change to the ports directory.

hostname# cd /usr/ports/security/sudo
hostname# make install clean && rehash
This will download the sudo source, compile, and install sudo. Now, I'll assume you'll want that second user you added to have root access, since this will be your main account from now on, so we'll need to edit the sudoers file.

hostname# cd /usr/local/etc
hostname# ee sudoers

Now, this will bring the ee text editor again. Find these lines:

# User privilege specification
root ALL=(ALL) ALL

Remember that second user we added during the install? Add the following under "root ALL=..." from above, but replacing username with whatever the username of the account that you added was.

username ALL=(ALL) ALL

Go ahead and save the file like we did before. Now we're ready to go headless, but before we do that, let's make sure you'll have access to the machine, before I tell you that you can remove your physical access to the box. Now run the command below:

hostname# sockstat

You should see sshd somewhere in the 2nd column. Now, you are ready to go to a headless system.

How to use SSH

I will quickly go over how to use SSH here for Windows and Unix. I will first start with Unix. On a unix machine you can open a console and run:

hostname# ssh username@ipaddress

If your username is the same as the one on the server, you do not need username@, but just the IP. If it is different, make sure username is replaced with the user you wish to log into the box with. To find the IP address of the old computer, you'll need to have physical access to the box. Run the following command on it after logging in.

hostname# ifconfig

You should see a screen similar to the following:

ifconfig

Your IP address should be in a location similar to the red circle and chances are that it will not be the same as the one in the picture. The IP address you want is going to be under the network device you configured during installation. From there on out, login is pretty obvious and it's just like as if you were sitting in from of the monitor. Use the command exit to close the ssh session.

Windows is a bit different because it does not have a built in client, so you will need to download one. I highly recommend PuTTY. On the downloads page, make sure you download putty.exe and not the other ones, because they lack SSH. Once downloaded, double click on the file. You should see a window come up like the one below. Note: Your fonts will not look as bad as they are in these screenshots. I am running PuTTY via Wine on FreeBSD because I do not have a Windows computer handy. :)

putty

After filling in the IP address, as in the screenshot above, you will see the following window pop up. Click Yes on the security alert. This just tells us that it hasn't cached the SSH key.

putty2

Below is an ugly example of PuTTY running. Once again, I am terribly sorry I don't have my fonts setup correctly in wine. It will look much better on native Windows. Here on out it's just like having physical access as well.

putty3

Office Utilities

So we know you'll eventually need to do some work -- thanks to the guy that said I'm going to work all of my life and save up money the whole time. First thing you will probably needs is an email client. I suggest mutt. It is an extremely nice and feature complete text mode email client. Make sure you have logged into your second user account for the rest of this article.

hostname# cd /usr/ports/mail/mutt && sudo make install clean && rehash

This will download the mutt source, compile, and install it, just like we did with sudo. I would like to cover the setup of mutt, but there is a great guide here that would definately out do me. :)

Now I know you'll need a calender sometime. Besides the default unix cal program there are many other alternatives. I'm going to install Pal for you.

The first thing you need to do is run the following commands:

hostname#cd /usr/ports/deskutils/pal && sudo make install clean && rehash

As with the install of sudo, this will download, compile, and install pal. From there, I suggest you run man pal and read what is in the man page. Here is a quick way to get started though:

hostname# pal -m

Pal -m is interactive mode. This allows you to add/remove events, and even set up notifications of events.

pal -m

Assuming you are like any other person, you will need some way to read your PDF and Word documents. How would you do this in text mode? Easy! We harness the power provided by antiword and pdftotext (part of the xpdf project).

To install antiword, run the following:

hostname# cd /usr/ports/textproc/antiword && sudo make install clean && rehash

This will install antiword and allow you to view your documents as a text document. This also has ample documentation once installed, so I won't cover its usage. Please run the command antiword to see a quick list of what to do, or run man antiword to see the full system documentation.

To install pdftotext, you need to run the following:

hostname# cd /usr/ports/graphics/xpdf/ && sudo make WITHOUT_X11=yes install clean && rehash

Make sure you include the WITHOUT_X11 or you will spend hours compiling the X server and all of it's dependencies. With pdftotext, you can convert PDF files to text or HTML documents to view in a web browser. I will not cover how to use pdftotext because you can just type in pdftotext and press enter at the prompt to get a quick how to. If you want more in depth information, run man pdftotext.

That brings us to our last office application, a web browser. Aren't we all spoiled by great applications like Firefox? I mean we have tabbed browsing, multiple download support, and good security. I bet you are thinking this is impossible in text mode, but you should know by now that it isn't. Elinks provides everything I just stated and is just as easy to use. To install Elinks, run the following commands:

hostname# cd /usr/ports/www/elinks && sudo make install clean && rehash

Once you have let it compile and install, you should be able to run the command elinks to open the browser. Once opened, it should look something like this:

Elinks

To start using it, just enter the address and press enter. Over ssh, you can usually use your mouse to interact with it, to some extent. If not, you'll need to use the up and down arrow keys. To open the main menu, press the escape key and use the arrow keys to navigate it.

Hopefully you understand all of the work that can be done in text mode and I'm only scratching the surface! Once you learn the application parameters and find new applications on your own, you will see how quickly you accomplish tasks.

Keeping in Touch - Chat

Besides using email, you are probably going to need a way to keep in touch with your friends and family. You have probably lost all hope in using any modern instant messenger in text mode, but once again, you should know better. I will introduce you to an application that allows you to use AIM and ICQ from within text mode. So let's get installing!

As usual, run the following command:

hostname# cd /usr/ports/net-im/naim && sudo make install clean && rehash

Once you have it installed, run the command naim and you should see the image below.

naim

It will take a while to get used to, but to login you need to type /connect yourScreenName and then you will need to type your password and press enter. You should also notice that the bar above where you type gives you hints. You should also familiarize yourself with the /help command.

If you would like to know more about the other features of naim, I suggest you read the man page as usual. Run the command man naim to see it.

For you geeks out there -- that means you. ;) You probably already know about this one, but I'm going to go over it anyway. The most famous text mode client of them all, irssi. Irssi is a full featured IRC client that supports multiple servers, multiple chat channels, perl scripting, theming, and many other features. So, why stop now? Let's get installing! The command is as you guessed it:

hostname# cd /usr/ports/irc/irssi && sudo make install clean && rehash

To use irssi all you need to do is run the command irssi. From there, you have all of your basic irc commands, plus a few irssi specific ones for controlling the window. Here is a run down on connection to an irc network. In this case, we'll use an irc network I helped found, The Wildcard.

First thing you need to do is run irssi and type /server irc.thewildcard.org. Once you are connected, type /join #main and /join #equality. This will put you in two channels. You can then switch between windows using /win window#. In this example, /win 1 would be the server window and /win 2 and /win 3 would be #main and #equality. Everything else is just as you would expect with an IRC client.
[IMG irssi]

Gaming

There are a ton of games that can be played in text mode. I am only going to show you how to set up a few smaller games, but if you are a more advanced user, you can compile SDL to use ASCII as its output mode. You can then plan Quake and any other SDL application from within a console. You may want to google libaa and libcaca for more information on doing this.

First of all, let’s get you set up with one of the most addicting text mode games that I have ever seen, Nethack. Nethack is an amazing game, it is as if the developers thought of almost every possible thing. I guess that is why it is still popular. So here goes…

hostname# cd /usr/ports/games/nethack34-nox11/ && sudo make install clean && rehash

Once it is done, that’s all there is to it! Just run the command nethack and go. You may want to read the man page (by running man nethack) or visit www.nethack.org for more information.

Nethack!

After you spend weeks playing Nethack, you may want to play some good old classics. There are plenty of those in text form. I’ll start off with nInvaders. This is by far one of the coolest text mode Space Invaders clone that I have ever seen. To install it run:

hostname# cd /usr/ports/games/nInvaders && sudo make install clean && rehash

Once installed, it’s pretty simple. Run the command nInvaders and use the right/left arrows to move and the spacebar to shoot.

nInvaders

If you want to go with another classic, you can play BSDtris. This is a full features ASCII Tetris clone. To install:

hostname# cd /usr/ports/games/bsdtris && sudo make install clean && rehash

As soon as that finishes up, all you have to do is run the command bsdtris and follow the key guides, on the screen, to play.

BSDtris

Wrapping it all up

I hope this has opened the world to some of you people afraid of text mode. There is a whole new world of software out there waiting for you to discover it. I suggest starting at sites like freshports.org and freshmeat.net and just start browsing. There are many more things that can be installed from the FreeBSD ports collection, but you should at least know the basics. If you have any questions, don’t hesitate to reply to this post or join us in IRC.

High Traffic Web Development

Sunday, April 16th, 2006

Over the past week and a half, I have been in the process of reading up on developing for high traffic sites. It is very interesting that it only requires small optimizations here and there.

One thing I’ve found in common with all large sites and that should be pretty obvious, is a PHP compiler cache and some type of caching system. Some of the more popular ones are APC, Bware Afterburner, Turck MMCache, and the Zend Accelerator. PHP caching systems are easy to write and can speed up template calls a ton.

You can speed up your site by using output buffering. What this does is use writev() instead of write(). The write() calls were sent to apache as 4kb buffers, where as the writev() calls aren’t.

Another speed up can be accomplished by setting Apache’s SendBufferSize to PageSize. This allows the page to be handed over to the kernel, to be sent, without blocking.

To reduce bandwidth, you may want to look into gzipping the contents of your page. I have seen this shrink 80gb backups down by almost a third the size, except for when most of the contents were images. So you should gain a lot for this if you can afford the overhead of the operations to compress your data.

There are also some simple thinks you can do to speed up your code. To speed up database calls, only query for that data you need. No need to SELECT * when you only need to SELECT id. You should also only query a table once. Get the data you need from it and store it, don’t query it again.

If you are using PHP5 or higher, you may also want to use MySQLi. This new MySQL API is much faster than the old API and includes both a functional and OOP interface. Most applications can be easily converted to use it. Plus, you get support for prepared statements and bind statements.

Optimization can also be made during the design of your database. Make sure you realize the differences between MyISAM and the InnoDB storage engines. MyISAM is very efficient for either very high volume writes or reads, but has table level locking. InnoDB has non-locking reads and row level updates, plus high concurrency.

You may also want to cache query results if they are not expected to update as often. An example I often use for this is CMS systems. Why query all of the blocks you want on one page, when you can cache the results of the original query for 6 hours and have them quickly accessible? Plus, if the owner of the site changes them, just clear the cache…

Now, back to what I’ve been up to. Besides reading up on the above, I have also been messing with my new project for the last two weeks straight. It is amazing how many hacks IE5-IE6 requires. I know I’ve used at least four to get my new site to display correctly. While the thing worked in Firefox, Safari, Epiphany, Konqeror, Opera, and IE7 the _whole_ time.

I hope most people will like the design though. I have shown it to around 20 people on IRC, and only one person hasn’t liked it a lot. So I would say I’ve done pretty good considering this was the first site I had designed entirely in Photoshop and then converted it to CSS/XHTML by hand. The core work for this new project starts this week and will probably continue for the next few months to a year.

Reference for high volume PHP: http://www.oreillynet.com/onlamp/blog/2006/04/digg_phps_scalability_and_perf.html

How do I roll? I roll with PHP/6.0.0-dev.

Thursday, March 23rd, 2006

I’ve had one insanely sweet idea for a website last Sunday. So after I got home today, I worked on building a php6 installed. I must say it’s not easy. I could not get the direct cvs checkout to compile correctly. It seems to me that flex and yacc weren’t generating the parser and I didn’t feel like fixing.

So I headed on over to snaps.php.net and downloaded a prebuilt cvs export. After installing at least 15 libabc-dev packages via apt-get, I was ready to build. So I started it and saw something about ICU. Well it turns out, the Ubuntu package isn’t up-to-date enough. PHP 6 requires ICU 3.4+. So I had to compile all 8.5mb of that code. Make sure that you run ./configure like this on ICU (according to the one thing I found on google about it, let me know if it works differently) ./configure –prefix=/usr/local/ –disable-threads
Here is what I build my PHP 6.0.0/dev install with:

./configure –program-suffix=6 –with-apxs2=/usr/local/apache2/bin/apxs –enable-openssl –with-zlib –enable-bcmath –with-bz2 –enable-calender –with-curlwrappers –enable-dba –with-inifile –with-flatfile –enable-exif –enable-ftp –with-gd –with-gettext –enable-mbstring –with-mcrypt –with-mhash –with-mime-magic –with-mysql –with-mysqli –with-unixODBC=/usr/ –with-readline –with-mm –enable-soap –enable-sockets –enable-sysvshm –with-tidy –with-xsl –with-icu-dur=/usr/local –disable-mbregex

Now, I’m off to mess with my long wanted namespaces in PHP.

Music hole on purevolume!

Saturday, January 28th, 2006

I was browsing purevolume.com and listening to albums with their flash music player. It got me wondering, does it stream the music or download to my computer? So I checked /tmp and found that they were in /tmp/plugtmp. Sure enough it downloads them in a hashed file without an extension or id3 tags. This works on the songs they don’t even allow you to download. I do not know if this does the same on windows or not.

Here is how I discovered this:

  1. Listened to the full song
  2. Wondered what I said above
  3. Checked /tmp/plugtmp
  4. Opened the hashed files in mplayer

NOTE: I do not take responsibility for anything you do or get in trouble for using this method of obtaining music. I fully support buying music but decided it would be good to get this out there so a good site doesn’t get sued by the RIAA.