I’ve got some code that builds a SQL query from a variable set of parameters. Most parameters are involved with building a dynamic WHERE clause. Other parameters are for pagination, setting a LIMIT clause on the query that eventually gets built. Another parameter is whether to make the query a COUNT() query or whether to actually pull rows from the db.

In the case where I build a COUNT() query, I was carelessly allowing a zero-based LIMIT clause - something like LIMIT 0,10 - to be appended to the query. No sweat, I figured, since I really only need the single-row containing the count I am requesting.

However, it turns out that this LIMIT was actually be applied *before* the COUNT()-ing, not after. So, my COUNT() was completely wrong, often returning 0.

There’s an hour of my life - spent in furious frustration - that I’ll never get back.

Onward!

I just read on AskWoody.com that Martin Gardner, the longtime author of the Scientific American column “Mathematical Games”, has died at the age of 95.

I am a bit surprised at how it is affecting me.

As a kid, I loved his column. I took pride in solving the puzzles. The activity taught me focus and concentration. It taught me problem solving. It taught me to think.

But most of all, it gave me my first exposure to the notions of logical incompleteness and unequal infinities.

With the naivety of youth, I believed that the universe was black and white. Statements were either true or false, and essentially all knowledge (using a sufficiently large book) could be expressed in language.

Further, infinity was infinity. the biggest thing there is. Nothing bigger than that, despite the common schoolyard escalation of taunts:

“You have cooties.”

“Well, you have cooties plus one.”

“You have cooties plus two.”

“Oh, yeah? You have cooties times one hundred”, cleverly upping the stakes by switching from simple addition to the more powerful multiplication.

“Well, you have cooties times infinity!”

“Oh yeah? You have cooties times infinity-plus-one.”

… and so on.

Still, infinity came to connote - at least to me - a sense of maximality, the sum totality of everything, of the universe. Of The Universe.

Martin Gardner’s column exposed me to Gödel’s Incompleteness Theorem and Cantor’s Theorem about cardinalities.

Gödel’s Incompleteness Theorem states that no consistent system of axioms whose theorems can be listed by an “effective procedure” (essentially, a computer program) is capable of proving all facts about the natural numbers. For any such system, there will always be statements about the natural numbers that are true, but that are unprovable within the system.

This has profound implications for reductionist attempts - to which I was inherently attracted in my youth - to reduce all knowledge to a set of indisputable, logically consistent axioms and deduction mechanisms. Essentially, if you depend upon axiomatic logic and consistency, you are limited in what you can prove. There exist statements that are expressible within the given system, but whose truth or falsity are not provable using only the tools of the system.

This result undermines language as the sole medium of defining knowledge, since language will always produce statements that are outside the reach of provable truth or falsity. It casts doubt on the binary notion of truth, opening up the mathematical discipline of fuzzy logic.

It even strikes closer to home for me, contradicting something my father explicitly told me when I was a child, one of his core beliefs: “If you can’t describe what you know, then you do not know it.”

Gödel’s method of proof was his famous Diagonal Lemma, in which he starts with the logical system, its symbols and its constructs, and then proceeds to build an example of his elusive unprovable statement.

Using a similar diagonalization technique, Georg Cantor was able to demonstrate that the infinity that represents the natural numbers (1, 2, 3, 4…) was a “lesser” infinity than the infinity representing the real numbers (the numbers that represent measurable distances on a line), the former being a “countable” infinity and the latter an “uncountable” one. Further, he proved that there are ever-higher, unlimited levels of “greater” infinities, all of which are “uncountable”.

All this just blew me away. It fired my imagination. It opened my mind to an extraordinary universe, to universes, to multiverses. It inspired me to embrace the beauty of abstraction. It led me to study mathematics, first as an undergraduate, then as a graduate student, through to my doctorate degree.

Abstraction, modeling, language, truth, philosophy, knowledge, infinities, alternate possibilities, universes.

For me, it all started with Martin Gardner. My life would not be the same without the contribution that he made to it. I’m saddened at the news of his passing, but grateful for the door he opened for me.

So, what’s the problem?

As of this writing, @ZodiacFacts has over 170,000 followers - a lamentable indictment on the state of human rationality. It is no surprise that some of them continue to retweet into the #zf stream.

There are not many of them, only a few each day. So, it’s only a minor annoyance. Still, life is filled with enough of minor annoyances that it’s desirable to remove/minimize them when we can.

The solution, perhaps

A Twitter Bot could be built that polls Twitter for tweets containing both @ZodiacFacts and #zf and then sends a politely worded reply to the Twitter user requesting that he refrain from using the #zf hashtag.

I have actually built a class on which such a bot can be based. And I have registered the Twitter account @zfisforzend to function as the sender.

But since the #zf hashtag should rightly be regarded as a community resource, I don’t really feel that it is my place to deploy the solution on my own. The “community” or even Zend HQ should make the decision about whether to deploy, the phrasing of the message, etc. After all, both of these things affect the public face of the project. Hey, it might even inflame the legions of @ZodiacFact fans to spam the #zf stream, just out of pique.

So, I am posting my current code so that if one of the project leaders wishes to use it - or to develop their own, which would no doubt be better than mine - then they can do so with little effort. I’d be happy to pass off the ownership of the @zfisforzend Twitter account to whatever community member wants to use it for this purpose.

/**
 * Access Twitter API, do twitter search, then reply
 *
 * @author David Weinraub ( david@papayasoft.com )
 */
class PS_TwitterSearchAndReplyBot
{
    /**
     * @var Zend_Service_Twitter
     */
    protected $_twitter = null;

    /**
     * @var string
     */
    protected $_reply = null;

    /**
     * Path to the file containing the id of the tweet to which we have replied
     *
     * @var string
     */
    protected $_sincePath = null;

    /**
     * @var string
     */
    protected $_userAgent = 'PapayaSoft-SearchAndReplyBot/1.0';

    /**
     * Constructor
     *
     * @param string $username Twitter username for the acct sending replies
     * @param string $password Twitter password for the acct sending replies
     * @param string $query the query we use to find tweets.
     * @param string $reply the text of the reply we wish to send
     * @param string $sincePath path to the file where we save the last replied tweet
     */
    public function __construct($username, $password, $query, $reply, $sincePath)
    {
        if ('' == $username){
            throw new Exception('User is required');
        }

        if ('' == $password){
            throw new Exception('Password is required');
        }

        $this->_twitter = new Zend_Service_Twitter($username, $password);
        $this->_twitter->getHttpClient()->setHeaders('useragent', $this->_userAgent);

        if ('' == $query){
            throw new Exception('Query is required');
        }
        $this->_query = $query;

        if ('' == $reply){
            throw new Exception('Reply is required');
        }
        $this->_reply = $reply;

        $this->_sincePath = $sincePath;
    }

    /**
     * Get the matching tweets
     *
     * @return array the matching tweets
     */
    public function getTweets()
    {
        $service = new Zend_Service_Twitter_Search();
        $service->getHttpClient()->setHeaders('useragent', $this->_userAgent);
        if (null == $sinceId){
            $sinceId = (int) $this->readSince();
        }
        $params = array(
            'since_id' => $sinceId,
        );
        return $service->search($this->_query, $params);
    }

    /**
     * Send the reply to a matching tweet
     *
     * @param array $tweet
     * @return void
     */
    public function sendReply($tweet)
    {
        $target = $tweet['from_user'];
        $reply = '@' . $target . ' ' . $this->_reply;

        $result = $this->_twitter->statusUpdate($reply, $tweet['id']);

        if ($result->isSuccess()){
            $this->writeSince($tweet['id']);
            return true;
        } else {
            throw new Exception('Failure posting status reply to target');
        }
    }

    /**
     * Save the id of the last tweet to which we have replied
     *
     * @param int $sinceId
     * @return 
     */
    public function writeSince($sinceId)
    {
        @$fp = fopen($this->_sincePath, 'w');

        if (!$fp){
            throw new Exception('Unable to open since-file for writing');
        }
        fwrite($fp, $sinceId);
        fclose($fp);
        return $this;
    }

    /**
     * Get the twitter id of the last tweet to which we have replied
     *
     * @return int
     */
    public function readSince()
    {
        return file_get_contents($this->_sincePath);
    }

    /**
     * Entry point. Do it all.
     */
    protected function run()
    {
        $tweets = $this->getTweets();
        foreach ($tweets as $tweet){
            $this->sendReply($tweet['from_user'], $tweet['id']);
        }
    }
}

Usage would be something along the following:

$bot = new PS_TwitterSearchAndReplyBot(
    'zfisforzend',
    'thepassword',
    '#zf AND @ZodiacFacts',
    'The ZF hashtag has been used by http://bit.ly/dlRih0 for 2+ years. Please do not use it for ZodiacFacts. Thanks! ;-)',
    dirname(__FILE__) .'/data/since.dat'
);
$bot->run();

Then set up a cron job to run this script with some polling frequency.

Note that the message we send should not contain either of the terms “#zf” or “Zend” directly. After all, those are search terms we all use in our Twitter clients. The last thing we want is for all these bot-generated replies to further pollute the targeted hashtag stream.

As you can see, it is general enough to serve as the base for other Twitter Reply bots. I confess that the code is only partially tested, but I think it conveys the idea.

So, whaddya think?

Using this structure usually requires creating a virtual host pointing to this public folder and then creating the file public/.htaccess that maps all non-file, non-directory requests to the index.php file. You then get to enjoy all the rich, creamy Zend goodness.

But sometimes you are stuck with shared hosting and you can’t always place your other app folders as siblings to your public web root. For example, one prominent web hosting company provides me a web root of the form /usr/www/users/myusername with a home directory of /usr/home/myusername.

So whaddya gonna do?

A Solution - Push Down and Protect

This question has been asked and answered many times, but most of the solutions use either custom plugins or tricky .htaccess rules. There’s nothing wrong with those - and in general, they are proposed by people way smarter than me. So there is an argument for going with the best. As they used to say: “No one ever got fired for buying IBM.”

But there is a simpler way that has worked well for me: depart slightly from the recommended file structure and modify some include paths. I have seen this approach referenced before, in particular by tharkun on Stackoverflow, but I had not seen it written out explicitly. So, I offer it here in the hope that other ZF n00bs like me can benefit from it.

Essentially, I just push all my app folders down into a subfolder of the web root called something like _zf, web protect that _zf folder with a Deny from All in an .htaccess file for the folder, and change a few paths in the index.php entry point.

I presume that the shared hosting offers me a web root folder called public_html.

So, my folder structure looks like this:

I make sure that the file public_html/_zf/.htaccess prevents any direct web access:

Deny from All

This keeps anyone from viewing any of the non-public app files.

Then we just modify paths in public_html/index.php to point to the _zf folder, as follows:

set_include_path(implode(PATH_SEPARATOR,
    array(
        dirname(__FILE__)  . DIRECTORY_SEPARATOR . '_zf' . DIRECTORY_SEPARATOR . 'library',
        get_include_path(),
    )
));

// Define path to application directory
defined('APPLICATION_PATH')
    || define('APPLICATION_PATH', realpath(dirname(__FILE__) . '/_zf/application'));

// Everything else is the same as usual. For example, using Zend_Application...

// Define application environment
defined('APPLICATION_ENV')
    || define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'production'));

require_once 'Zend/Application.php';  

$application = new Zend_Application(
    APPLICATION_ENV,
    APPLICATION_PATH . '/configs/application.ini'
);
$application->bootstrap()->run();

And that’s it!

A purely modular app structure can also be handled in this way. For example, you can simply put your modules folder inside your application folder. Since all the logic associated to module routing, module bootstrapping, and module-specific paths already resides within the framework, the setup above still works.

The Downside

There is an insidious down-side to all this. Best practice dictates keeping all those app folders - and especially the config files, rich with passwords and the like - out of the web root. In theory, we have that all handled with the .htaccess Deny on the _zf folder. But it’s a single point of failure. If it fails for some reason - say, the .htaccess file doesn’t get deployed - then we are fully exposed to a wide variety of potential mischief.

There is some additional obscurity you can obtain by changing the name of the _zf folder to something (practically) unguessable. I could even imagine a cron job that renames the folder and makes corresponding changes to the index.php file. But that’s probably excessive. And, in any case, as we should all well know, relying solely on security through obscurity is a poor approach.

But maybe this approach is useful to get an app off the ground with a customer who is stuck with his shared hosting package. At some point in the future, when the app is sufficiently profitable, maybe you can convince the customer to move to virtual private server hosting, at which point it’s a small modification to return to the standard ZF-recommend file structure.

Whaddya think?

In particular, one of Netbeans’ nice features is Code Templates, little snippets of text that you can insert into your document simply by typing a short prefix and then hitting the Tab key. They are actually smarter than just static snippets in that they can contain variables that are dynamically filled/changed as you edit. Hugely useful. I use them constantly for a variety of tasks, but my most common use of code templates is for debugging.

One useful debug tool is FirePHP in conjunction with Firebug, especially for applications using AJAX.

Several weeks ago, Jeremy Kendall (@JeremyKendall) helpfully pointed me to Zend_Debug::dump() which behaves like an enhanced var_dump(). I was actually gratified to see that it resembles one of my own hacks that I have been using for years, though Zend_Debug::dump() is, not surprisingly, more polished.

Following on that, Markus (@derstahlstift) posted a Netbeans code template that uses Zend_Debug::dump().

I admit that they are both cool, but I prefer to identify precisely where I am doing the dump, using PHP’s magic constants __FILE__ and __LINE__. This allows me to easily chase down the dump statements and remove them once the debugging is finished.

So, here are my own versions, both in the form of Netbeans Code Templates.

For FirePHP::send():

FB::send("${VAR default="variable"} = " . $$${VAR} . ":: ${func default="func"} :: " . __FILE__ . "(" . __LINE__ . ")");
${cursor}

For Zend_Debug::dump():

Zend_Debug::dump($$${VARIABLE variableFromNextAssignmentName default="variable"}, "${VARIABLE variableFromNextAssignmentName}", true);
echo "

" . __FILE__ . "(" . __LINE__ . ") :: ${function default="function"} :: ${message default="message"}

";
${selection}${cursor}

Whaddya think? Do you have any tricks/shortcuts that you use for debugging? Let me know.

Cheers!

Since I have recently begun using Zend Framework for much of my development, I wrote a little view helper to handle it.

class App_View_Helper_HeadTitle extends Zend_View_Helper_HeadTitle
{
    public function headTitle($title = null, $setType = Zend_View_Helper_Placeholder_Container_Abstract::APPEND)
    {
        if (null !== $title && APPLICATION_ENV == 'development'){
            $title = strtolower($title);
        }
        return parent::headTitle($title, $setType);
    }
}

To use it, just add the helper path to your bootstrap. For example,you could do it in your config/application.ini with:

resources.view[] =
resources.view.helpers.App_View_Helper_ = "App/View/Helper"

The calls to $this->headTitle() in your view scripts or layout scripts remain unchanged.

As a side note, a great repository of Zend Framework snippets is at the aptly named site ZFSnippets.com.

Do you use any similar tricks or snippets in your development? Let me know via the comments.

Cheers!

[As a side note, I should have done the upgrade ages ago. But I had so many local databases on my server, and though an in-place upgrade - just copy the data directory of the old version to the new version - is possible within a series, it is not advisable when jumping to a new series. The alternative is to export/dump the databases and then re-import. I certainly could have used phpMyAdmin, but since I had so many local databases on my server, it would have been obnoxious to do it that way. I finally got around to hacking together a little backup script that queries the server for all its databases and calls mysqldump on each one. I also wrote an analogous restore script that was used to import the dumped databases into the new server.]

So, after backing up everything, and removing the 4.1 installation - including a manual deletion of all the orphan files that the uninstaller leaves behind, as well as a manual hunt-and-destroy of all MySQL references in the registry - I set about installing the 5.1 version. All smooth, right up until the last step: Configuring the server and installing the daemon as a service using the Instance Configuration Wizard.

The wizard asks a bunch of questions about maximum concurrent connections, transaction support, multi-language support, all straightforward. But at the end, it wants to create a new root passwords and before it lets you do that, it asks for the old ones.

The sticky part was that no matter what I did, I could not get the Wizard to accept my new root password. I find this pretty goofy, since this is supposed to be a brand new install. Even worse, I find it mysterious. How does it even know that there are old passwords out there? Didn't I delete everything?

But sometimes, you don;t have the luxury of contemplating the mysteries of the universe. Deadlines to meet. Mouths to feed. Plow ahead.

Apparently, I was not correctly reporting the old root password. I hardly ever use the root password, just generally safer to use a limited-privileges user account.

And no matter what I did - repeated uninstalls, manual deletes, restarts, etc - nothing could get past this critical step.

A bit of searching came up with a bunch of folks complaining in a MySQL bug reportof a similar issue, but none of the solutions there worked for me. The hero of the day was Jay Alverson, who comments on a MySQL forum that you need to delete the folder:

C:\Documents and Settings\All Users\MySQL

Turns out that all my old databases were there, including the mysql database that contains all the users, including the root user, and their passwords. Once that bad boy was gone, the wizard stopped asking me for the old root password, and simply allowed me to set the new one. Whew!

Hope it helps.

Siam Real Estate, a Phuket-based real-estate agent specializing in Phuket property will soon launch Siam Villa Rentals.

At present, only a placeholder page. But stay tuned.

But now that I am getting some trackbacks and pingbacks, I notice that the comment handling is missing something that I generally value in a blog. When listing comments, I prefer the real, human comments to be separated from the pingbacks/trackbacks. Although the pingbacks/trackbacks are completely important for their SEO value, I find that they can interrupt the flow of the conversation, disrupting the continuity. In my opinion, it’s better to list them separately.

Fortunately, it’s pretty easy to do. The theme file comments.php usually contains a loop that renders all the comments. The core of the technique is to use the Wordpress function get_comment_type() to determine the type of comment, then build two distinct buffers, one for which get_comment_type() returns ‘comment’, another for all the others (the trackbacks and pingbacks).

More details - though using a slightly different overall approach - can be found at Ryan J. Parker’s blog which, ironically, uses a modified version of the Fresh theme and where I suspect that this very post itself will appear as a pingback, nicely separated from the real human comments.

The rendered widget appears in the browser as a simple select dropdown with options for all the language supported by Google Translate. When the user selects his desired target language, the widget is supposed to contact the Google Translate mother ship, translate the text on the page, and then add an fixed iframe panel at the top of the browser viewport, followed by the translation of the page. The translated page even implements onmouseover handlers for text-based elements that display the original source text. Sweet.

It has worked great for me in the past. But I recently did a new deployment and I was unable to get the widget to work. When I (as the user) selected my target language, I got the following:

Error: The server could not complete your request. Try again later.

A bit more poking showed that it was only happening in Firefox. All my other browsers were ok. Eventually, I narrowed it down to a conflict with the Flashblock extension. Disabling the extension solved the problem.

Now, the tough choice is to run without the Flashblock or without the Google Translate. But at least I can deploy for the customer.

2010-02-20 Update: The mere presence of the enabled Flashblock extension when visiting a page with the translate widget does not cause the problem. The issue only occurs when the page has Flash content and the extension is configured to block Flash on that page.

The reason: The widget appears to actually use Flash!

In the file

http://translate.googleapis.com/translate_static/js/element/main.js

the function wf() seems to add Flash embed code into the page. I imagine that Flashblock is detecting this attempted insertion and is doing its magic.

I actually find it interesting - and impressive - that the Flashblock extension is smart enough to not only find/block Flash content on initial page load, but also at any time after that. I imagine it monitors the DOM and is vigilantly swaps out any Flash embeddings with its own replacement button.

I can see that the widget creates several iframes, at least one of which has Flash content pulled from the domain translate.googleapis.com. It stands to reason - well, at least to me - that enabling Flash content for this domain should do the trick. But so far, no luck. ;-(