Setting up a Web Server on AIX 6.1

There have been several occasions when I have needed to set up web servers on either Windows or Linux.  Pretty much every time it has been a simple process of installing binaries through a GUI and making sure everything plays nice together.  Well, for the first time I needed to set up Apache, PHP and MySQL on an AIX machine that is completely GUI-less (Note: the GUI-less portion isn't the hard part).  In going through this process I learned a few things, so I thought that I would share them here.

Step One: Trying to find binaries for AIX
This step in itself is a little difficult.  Previously I have used Bull Freeware (http://www.bullfreeware.com/) for AIX binaries.  However, in this case, most of the binaries available are fairly outdated and I really wanted to put something together that was a little more recent.  This drove me to the discovery of PmWiki (http://www.perzl.org/aix/index.php) maintained by Michael Perzl.  This repository of RPMs built for AIX is very thorough, and very recent!  I highly recommend this site for anyone looking for binaries for AIX in general.  I quickly snatched up binaries for Apache and PHP, but no MySQL binaries were available.  Have no fear, MySQL actually maintains their own, so I was able to download them from their website (http://dev.mysql.com/downloads/mysql/).

Step Two: Trying to install the binaries
This quickly turned into a dependency hunt for both Apache and PHP, but the PmWiki site had every package that I needed, so although tiring, it was simple work.... especially since each RPM install tells you exactly which packages it's missing.  Installing MySQL was as simple as unzipping the directory to /usr/local/mysql and following a few quick steps found in the install readme included with the binaries.  At this point, Apache was up and running, PHP was configured and MySQL was up and running.  In order to get MySQL to run at system startup I copied the mysql.server script to /etc/rc.d/init.d and then created symbolic links to it under rc2.d, rc3.d and rc5.d.  (Note: If for some reason, you are following this like a tutorial, do not install the PHP binaries just yet.... you'll see why).

Step Three: Trying to get PHP to connect to a MySQL database
This is when things started going downhill a little bit.  In a quick test, I got an error in my PHP script noting that the mysqli_connect (and mysql_connect) functions were undefined... this is not a good sign.  In going through the PHP info, I discovered that the PHP binary I downloaded was not compiled with MySQL support.  Bummer -- this meant that I now needed to compile PHP on my own on AIX.  Fun stuff!

Step Four: Compiling and Installing PHP on AIX
This part was the most painful of this entire process.  I don't have too much experience building and compiling from source (especially on AIX), so it probably took a little bit longer than it should have.  I did discover a few interesting things though:

• You must use the 32-bit version of MySQL if you want PHP to compile using it
• You must use gmake/GNU make instead of the AIX make command for compilation to complete
• I had to install the apxs (from the httpd-devel package) to reference in my configure command.  The AIX one (under /opt/pware64) made a libphp5.so library that caused Apache to fail to start up.
• The --enable-maintainer-zts configure option is required to make php thread safe.  Without it, Apache complains and won't load the library.
• I also came across a very odd problem when I did the "make install".  The first part of the script deletes libphp5.so and then subsequently tries to call chmod on it, at which point it bombs out since it can't find the file.  Nice.  I came across a hack that also worked for me: Have two windows open, one ready to do the make install, and one ready to copy libphp5.so to the directory the script is expecting it to be in.  Start the make install, and then as soon as it has tried to delete libphp5.so, copy it where the script expects it to be.  By doing this, the file will be there when it's looking for it.  It requires a little bit of timing, so it may take a few tries to get it right.

For anyone interested, this is the configure line I used: 

configure '--cache-file=../config.cache' '--prefix=/opt/freeware' '--with-config-file-path=/opt/freeware/etc' '--enable-shared' '--enable-static' '--without-pear' '--with-gd=/opt/freeware' '--with-openssl=/opt/freeware' '--with-zlib' '--with-bz2' '--with-curl=/opt/freeware' '--with-t1lib=/opt/freeware' '--with-freetype-dir=/opt/freeware' '--with-jpeg-dir=/opt/freeware' '--with-png-dir=/opt/freeware' '--with-xpm-dir=/opt/freeware' '--with-zlib-dir=/opt/freeware' '--enable-soap' '--enable-bcmath' '--enable-ftp' '--with-iconv' '--enable-dom' '--enable-json' '--with-pcre-regex=/opt/freeware' '--with-apxs2=/opt/freeware/sbin/apxs' '--with-mysql=/usr/local/mysql' '--with-mysqli=/usr/local/mysql/bin/mysql_config' '--enable-maintainer-zts'

Step 5: Testing and Completion

At this point, I now had Apache running with PHP with support for MySQL.  I happily connected to the database and then handed the server over to another developer who would actually be using it.  This was definitely a learning experience, so I hope someone finds this information useful.

UPDATE: I also discovered that by default, cgi scripts were not executing.  This was due to the file permissions set on the httpd log file directory.  So, if you also run into this problem, just run a chmod 775 on your httpd log file directory (mine was located at /var/log/httpd).

Diving Into CakePHP and Ruby on Rails

I love web development. It's the one kind of programming that seems to continually intrigue me and bring me back for more. I developed my first site using PHP (essentially on the WAMP stack) about 8 or 9 years ago, and haven't looked back. It wasn't until recently that I started looking into frameworks and libraries such as jQuery, Prototype, etc. I always preferred doing things with a text editor from scratch. Sure, it took longer, but I new the complete ins and outs of all of my website. Well, now that time is becoming more and more of a luxury, I can't afford to build sites from scratch anymore. Hence the discovery of two great frameworks: Ruby on Rails and CakePHP.

First, if you don't have any exposure to either of these technologies, there are some similarities in their paradigms. Both are designed to save you, the developer, time in building websites. Their goal is to do the monotonous, repetitive, underlying work for you so you can move on to build the rest of the website quickly. both use principles of MVC (Model View Controller) development. Essentially, their design forces you to use good design principles in your website. A very basic, and incomplete, description of the MVC components could be the following:
A Model is the data and associated interactions (Objects and Database)
A View is the presentation of that data (HTML, CSS)
A Controller is the logic used (Ruby/PHP)
A more complete explanation can be found here.
Another great thing that both provide is the ability to use REST APIs. So, for example, if you were to collect data that you wanted to open up to third-party applications, you could provide them with an API to access it. To me, this is a must to allow for layers of abstraction I previously mentioned in another post.

Ruby on Rails, until very recently, was a mythical and powerful creature to me. I had heard great things about it, heard what it could do, but I had absolutely no experience with it. I decided to change that this week. Within minutes I had completed the download and install of all required software to be up and running (I already had XAMPP on my machine from other development projects). Two great tutorials for learning basic Ruby on Rails can be found here and here. I went through both of them and feel like I have a good understanding of the design principles and capabilities of the technology. Ruby is a very interesting language and pretty easy to pick up. Ruby on Rails seems to be picking up steam in the US as of late, and is being used by websites such as Twitter.

Now, at first I was thinking, why learn another language like Ruby? I already have a good grip on PHP, so why not just stick with that? Great question! (Yes, sometimes I answer my own questions.) That's why I first looked at using CakePHP (plus, my friend was interested in learning it as well). According to the creator of CakePHP, he liked the idea of Ruby on Rails and wanted to bring that to the world of PHP, so he basically took the idea. To quote him from here:
"While it's difficult to copy Rails in PHP, it's quite possible to write an equivalent system. I like the terseness of Ruby code, but I need the structure that Rails provides, how it makes me organize my code into something sustainable. That's why I'm ripping off Rails in Cake."
I followed along with the tutorials on the CakePHP website and had no problem setting this environment up either. I just dropped it into my existing XAMPP setup and ran with it.

I also found plenty of "CakePHP vs. Ruby on Rails" type discussions. In the end, I surprisingly don't have much of a preference. Both were easy to set up, both were easy to learn the basics of, and both provide the framework principles I was looking for. I have a feeling that with more exposure I'll be able to choose a side more easily, but in the meantime, I'm just waiting around for a real project to try these out on. What about you? Which do you prefer?

How to Use the Arc90 PHP API to Get Info From Twitter

I recently decided, as a programmer and self-proclaimed web developer, to dive into the Twitter API, mostly just for fun.  From the main Twitter API Page you can access a series of prebuilt libraries for various programming languages, which provide wrappers around the main Twitter API.  With PHP being my web developing language of choice, I quickly glanced over the libraries available, with one clearly standing out: Arc90.  Not only is it maintained by a fellow "Matt," it also provides access to mostly anything you could think of in the world of Twitter.  By the end of this article, you should be able to pull all of your followers from Twitter and list some basic data about them.

Installation and Environment: The main page for Arc90 provides a very basic introduction on how to install and use the package, which is sufficient for getting your PHP environment ready to use.  So, for the purpose of this, I will assume that your PHP environment is all set, your includes are in order and cURL is ready to go.  If you need further help with any of those areas, feel free to leave a comment below.  So, I will assume you have a set up like the following:

1. <?php  
2.   
3. require_once('Arc90/Service/Twitter.php');  
4.   
5. $twitter = new Arc90_Service_Twitter('username', 'password');  
6.   
7. try  
8. {   
9.     // Gets the users following you, response in JSON format  
10.     $response = $twitter->getFollowers();  
11.   
12.     // Store the JSON response  
13.     $theData = $response->getData();
14.   
15.     // If Twitter returned an error (401, 503, etc), print status code  
16.     if($response->isError())  
17.     {  
18.         echo $response->http_code . "\n";  
19.     }  
20. }  
21. catch(Arc90_Service_Twitter_Exception $e)  
22. {  
23.     // Print the exception message (invalid parameter, etc)  
24.     print $e->getMessage();  
25. }  
26. 
    
27. ?>

Note a few changes: Obviously, on line 5 you will need to replace 'username' and 'password' with your login credentials.  On line 10, I went ahead and removed the parameter to request an XML response.  I saw that the default was JSON and wanted to try that (since that is another new technology for me).  I'm also using the getFollowers() API call instead.  On line 13, I'm storing the returned data for future use. 

Important Note Regarding API Rate Limits: If you are unfamiliar with Twitter and its associated API calls, you need to make sure that you don't overdo it on the number calls you make.  By default, Twitter allows you to have 100 API calls per hour (this includes using third party applications like TweetDeck or Nambu), so you'll need to keep an eye on how many calls you're using.  If you go over that limit, you can potentially be blacklisted -- which makes no one happy.  You have a couple of options: 1) Request to be whitelisted using this form.  This will allow you to use 20,000 requests per hour, or 2) Watch your rate limit using a built-in API method which monitors your status, and conveniently does not cost you an API call to use.  You can find more info regarding limits, whitelisting and blacklisting here.  Also, during development, I've gone and saved the output to a particular API call to a text document that I can parse while I test.  It doesn't provide real-time data, but it allows me to work on it as many times as I want without a single API call.

Using the Data: Now that we have our data response in JSON format to work with, let's get to it! To make good use of the data returned, I chose to put it in an associative array for organized access.  To do this, we use the json_decode method built into PHP as follows:
$dataArray = json_decode($theData, true);
To view an example of the kind of data that a getFollowers API call will return, we can view this method page in the documentation.  Also, for a detailed description of each key that can possibly be returned from any of the various API calls, you'll want to keep the complete list handy.  I ended up making a spreadsheet with the various API calls that I tested along with the associated return values, for a quick reference -- so you may want to do the same.

Let's say that we just want to loop through our followers (or at least the first 100) and print out some data about each one, line by line.  All we have to do is iterate through the array, pulling out the data we want and display it.  I stuck with a basic HTML table for this, but all you CSS gurus out there can have way more fun with this.  This assumes that you already have your associative data array, dataArray, ready to go:

1. <html>
2.         <head>
3.             <title>Test</title>
4.         </head>
5.         <body>
6.             <h1>Followers</h1>
7.            <table border="1">
8.                 <tr>
9.                     <td>Image</td>
10.                     <td>Screen Name</td>
11.                     <td>Name</td>
12.                     <td>Description</td>
13.                     <td>URL</td>
14.                     <td>Location</td>
15.                     <td>Following</td>
16.                     <td>Followers</td>
17.                     <td>Status</td>
18.                     <td>Status Count</td>
19.                 </tr>
20. <?php
21.     $numItems = count($dataArray);
22.     for($i = 0; $i < $numItems; $i++){
23.         $currentItem = $dataArray[$i];
24.        
25.         $screenName = $currentItem["screen_name"];
26.         $description = $currentItem["description"];
27.         $followersCount = $currentItem["followers_count"];
28.         $url = $currentItem["url"];
29.         $name = $currentItem["name"];
30.         $status = "";
31.         if(isset($currentItem["status"]))
32.         {
33.             $status = $currentItem["status"]["text"];
34.         }
35.         $friendsCount = $currentItem["friends_count"];
36.         $profileImage = $currentItem["profile_image_url"];
37.         $location = $currentItem["location"];
38.         $statusCount = $currentItem["statuses_count"];
39.          echo "<tr><td><a href='http://www.twitter.com/".$screenName."'><img height='48' width='48' src='".$profileImage."'></a></td><td>".$screenName."</td><td>".$name."</td><td>".$description."</td><td>".$url."</td><td>".$location."</td><td>".$friendsCount."</td><td>".$followersCount."</td><td>".$status."</td><td>".$statusCount."</td></tr>";
40.     }
41. ?>
42.     </table>
43. </body>
44. </html>

 You should end up with something that looks like:

If you have any trouble working through this, or have any questions, just let me know!

Building PHP on AIX

This is more for my own benefit than anything else (or for anyone who stumbles across this through Google, or whatever search engine you prefer). I had quite the time building PHP on AIX, so I just wanted to chronicle what I did, including getting the PHP Excel package to work.

I roughly followed the steps for installing PHP on AIX available at:
http://www.ibm.com/developerworks/wikis/display/WikiPtype/aixopen

That walked me through at least installing prerequisites, and any other Linux/Gnu tools I may want.

Now, I just wanted the command line executable, so I skipped Apache all together. So, I did the following to build it:
export PATH=/opt/freeware/bin:$PATH
export CC=xlc
export CXX=xlc++
./configure --prefix=/usr/local/php --with-gd --with-zlib-dir=/opt/freeware --enable-shared --enable-debug --enable-zip --disable-static --with-zlib --with-bz2 --with-jpeg-dir=/opt/freeware --with-png-dir=/opt/freeware --with-xpm-dir=/opt/freeware --with-freetype-dir=/opt/freeware
ulimit -S -d unlimited
gmake
gmake install

In case you're wondering, the ulimit command removes restrictions on how much memory can be used by processes in the current session. I also had to use gmake. With make, it just failed over and over and over again.

Now, the whole point of installing php was so that I could create Excel xlsx files (which can go beyond the line limit in Office 2003), so I used the current release found at:
http://www.codeplex.com/PHPExcel

Since I was building extremely large spreadsheets, I had to update the memory limit in php.ini (and I upped the max execution time for good measure). But, even with this, I was hitting memory caps. So, the first thing I did was use the same ulimit command (mentioned above) before executing the script. This helped with the medium-sized spreadsheets, but with the large ones I was getting a segmentation fault.

After a little research, I discovered the following page:
http://www-01.ibm.com/support/docview.wss?rs=639&context=SSTFW4&dc=DB520&dc=DB560&uid=swg21302795&loc=en_US&cs=UTF-8&lang=en&rss=ct639tivoli

It explains why, when using XML (which the 2007 format uses), it can cause issues with processes that hog memory. So, I just took the solution from the page, which was to use the following command (in addition to the ulimit command) before calling the php script:
export LDR_CNTRL=MAXDATA=0x80000000

It was a mess, but I'm pretty sure it'll work now -- regardless of file size! It just takes awhile. The last test I did with 10 columns and around 66,000 rows took about 25 minutes and 800MB of memory to complete! Now if only PHPExcel would work on the amount of memory required (which I believe they are) we'll be in business.

Text-Based Games...

So, I have made it a life goal of mine to create a 3D game of some kind at some point in my life. Realizing the size of this daunting task, I've decided to start a little bit smaller. I think that I want to make a web-based, text-based game that's kind of like a choose your own adventure. I figure that it'll be a good place to start... see how it goes and then move forward from there.

In any case, my plan is to use PHP/MySQL to accomplish this, since I think it will work, but mostly because it's the two web-based technologies (besides HTML) that I'm most comfortable with. Does anyone have any other suggestions that might make my journey a little bit easier?

If I make some decent progress, I'll be sure to post updates here. Or, if it's something you think you'd want to participate in, let me know as well...

PHP is Magical?

I was looking through the differences between PHP4 and PHP5 this morning, when I noticed something amazing. PHP5 is magical! The screenshot on the left is taken directly from the PHP5 manual, and can be viewed here if you really don't believe me. I love how the last sentence warns you that you can't have these functions in your class "unless you want the magic functionality associated with them." My question is, who wouldn't want this magical power within their script? I know I would.