PapayaSoft - Phuket web development

What to Do About ZodiacFacts and #zf on Twitter?

| Comments

The Twitter hashtag #zf has been used by the Zend Framework for several years now. The Twitter user @ZodiacFacts began using it some time ago, polluting the #zf stream with all that medieval nonsense. Zend Framework Project Lead Matthew Weier O’Phinney tweeted a reply to @ZodiacFacts politely requesting that he/she refrain from using the hashtag and @ZodiacFacts seems to have politely complied.

So, what’s the problem?

As of this writing, @ZodiacFacts has over 170,000 followers – a lamentable indictment on the state of human rationality. It is no surprise that some of them continue to retweet into the #zf stream.

There are not many of them, only a few each day. So, it’s only a minor annoyance. Still, life is filled with enough of minor annoyances that it’s desirable to remove/minimize them when we can.

The solution, perhaps

A Twitter Bot could be built that polls Twitter for tweets containing both @ZodiacFacts and #zf and then sends a politely worded reply to the Twitter user requesting that he refrain from using the #zf hashtag.

I have actually built a class on which such a bot can be based. And I have registered the Twitter account @zfisforzend to function as the sender.

But since the #zf hashtag should rightly be regarded as a community resource, I don’t really feel that it is my place to deploy the solution on my own. The “community” or even Zend HQ should make the decision about whether to deploy, the phrasing of the message, etc. After all, both of these things affect the public face of the project. Hey, it might even inflame the legions of @ZodiacFact fans to spam the #zf stream, just out of pique.

So, I am posting my current code so that if one of the project leaders wishes to use it – or to develop their own, which would no doubt be better than mine – then they can do so with little effort. I’d be happy to pass off the ownership of the @zfisforzend Twitter account to whatever community member wants to use it for this purpose.

/**
 * Access Twitter API, do twitter search, then reply
 *
 * @author David Weinraub ( david@papayasoft.com )
 */
class PS_TwitterSearchAndReplyBot
{
    /**
     * @var Zend_Service_Twitter
     */
    protected $_twitter = null;

    /**
     * @var string
     */
    protected $_reply = null;

    /**
     * Path to the file containing the id of the tweet to which we have replied
     *
     * @var string
     */
    protected $_sincePath = null;

    /**
     * @var string
     */
    protected $_userAgent = 'PapayaSoft-SearchAndReplyBot/1.0';

    /**
     * Constructor
     *
     * @param string $username Twitter username for the acct sending replies
     * @param string $password Twitter password for the acct sending replies
     * @param string $query the query we use to find tweets.
     * @param string $reply the text of the reply we wish to send
     * @param string $sincePath path to the file where we save the last replied tweet
     */
    public function __construct($username, $password, $query, $reply, $sincePath)
    {
        if ('' == $username){
            throw new Exception('User is required');
        }

        if ('' == $password){
            throw new Exception('Password is required');
        }

        $this->_twitter = new Zend_Service_Twitter($username, $password);
        $this->_twitter->getHttpClient()->setHeaders('useragent', $this->_userAgent);

        if ('' == $query){
            throw new Exception('Query is required');
        }
        $this->_query = $query;

        if ('' == $reply){
            throw new Exception('Reply is required');
        }
        $this->_reply = $reply;

        $this->_sincePath = $sincePath;
    }

    /**
     * Get the matching tweets
     * 
     * @return array the matching tweets
     */
    public function getTweets()
    {
        $service = new Zend_Service_Twitter_Search();
        $service->getHttpClient()->setHeaders('useragent', $this->_userAgent);
        if (null == $sinceId){
            $sinceId = (int) $this->readSince();
        }
        $params = array(
            'since_id' => $sinceId,
        );
        return $service->search($this->_query, $params);
    }

    /**
     * Send the reply to a matching tweet
     * 
     * @param array $tweet
     * @return void
     */
    public function sendReply($tweet)
    {
        $target = $tweet['from_user'];
        $reply = '@' . $target . ' ' . $this->_reply;

        $result = $this->_twitter->statusUpdate($reply, $tweet['id']);

        if ($result->isSuccess()){
            $this->writeSince($tweet['id']);
            return true;
        } else {
            throw new Exception('Failure posting status reply to target');
        }
    }

    /**
     * Save the id of the last tweet to which we have replied
     *
     * @param int $sinceId
     * @return <type>
     */
    public function writeSince($sinceId)
    {
        @$fp = fopen($this->_sincePath, 'w');

        if (!$fp){
            throw new Exception('Unable to open since-file for writing');
        }
        fwrite($fp, $sinceId);
        fclose($fp);
        return $this;
    }

    /**
     * Get the twitter id of the last tweet to which we have replied
     *
     * @return int
     */
    public function readSince()
    {
        return file_get_contents($this->_sincePath);
    }

    /**
     * Entry point. Do it all.
     */
    protected function run()
    {
        $tweets = $this->getTweets();
        foreach ($tweets as $tweet){
            $this->sendReply($tweet['from_user'], $tweet['id']);
        }
    }
}

Usage would be something along the following:

$bot = new PS_TwitterSearchAndReplyBot(
    'zfisforzend', 
    'thepassword',
    '#zf AND @ZodiacFacts',
    'The ZF hashtag has been used by http://bit.ly/dlRih0 for 2+ years. Please do not use it for ZodiacFacts. Thanks! ;-) ',
    dirname(__FILE__) .'/data/since.dat'
);
$bot->run();

Then set up a cron job to run this script with some polling frequency.

Note that the message we send should not contain either of the terms “#zf” or “Zend” directly. After all, those are search terms we all use in our Twitter clients. The last thing we want is for all these bot-generated replies to further pollute the targeted hashtag stream.

As you can see, it is general enough to serve as the base for other Twitter Reply bots. I confess that the code is only partially tested, but I think it conveys the idea.

So, whaddya think?

Zend Framework on Shared Hosting

| Comments

Zend Framework recommends – but does not rigidly enforce – a file structure for your application. Most of the prominent “getting started” sample apps – like the Zend Framework Quickstart, Rob Allen’s Getting Started tutorial, HariKT’s Base Files skeleton – use it:

ZF recommended structure

Using this structure usually requires creating a virtual host pointing to this public folder and then creating the file public/.htaccess that maps all non-file, non-directory requests to the index.php file. You then get to enjoy all the rich, creamy Zend goodness.

But sometimes you are stuck with shared hosting and you can’t always place your other app folders as siblings to your public web root. For example, one prominent web hosting company provides me a web root of the form /usr/www/users/myusername with a home directory of /usr/home/myusername.

So whaddya gonna do?

A Solution – Push Down and Protect

This question has been asked and answered many times, but most of the solutions use either custom plugins or tricky .htaccess rules. There’s nothing wrong with those – and in general, they are proposed by people way smarter than me. So there is an argument for going with the best. As they used to say: “No one ever got fired for buying IBM.”

But there is a simpler way that has worked well for me: depart slightly from the recommended file structure and modify some include paths. I have seen this approach referenced before, in particular by tharkun on Stackoverflow, but I had not seen it written out explicitly. So, I offer it here in the hope that other ZF n00bs like me can benefit from it.

Essentially, I just push all my app folders down into a subfolder of the web root called something like _zf, web protect that _zf folder with a Deny from All in an .htaccess file for the folder, and change a few paths in the index.php entry point.

I presume that the shared hosting offers me a web root folder called public_html.

So, my folder structure looks like this:

ZF application structure for shared hosting

I make sure that the file public_html/_zf/.htaccess prevents any direct web access:

Deny from All

This keeps anyone from viewing any of the non-public app files.

Then we just modify paths in public_html/index.php to point to the _zf folder, as follows:

set_include_path(implode(PATH_SEPARATOR, 
    array(
        dirname(__FILE__)  . DIRECTORY_SEPARATOR . '_zf' . DIRECTORY_SEPARATOR . 'library',
        get_include_path(),
    )
));

// Define path to application directory
defined('APPLICATION_PATH')
    || define('APPLICATION_PATH', realpath(dirname(__FILE__) . '/_zf/application'));

// Everything else is the same as usual. For example, using Zend_Application...

// Define application environment
defined('APPLICATION_ENV')
    || define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'production'));

require_once 'Zend/Application.php';  

$application = new Zend_Application(
    APPLICATION_ENV, 
    APPLICATION_PATH . '/configs/application.ini'
);
$application->bootstrap()->run();

And that’s it!

A purely modular app structure can also be handled in this way. For example, you can simply put your modules folder inside your application folder. Since all the logic associated to module routing, module bootstrapping, and module-specific paths already resides within the framework, the setup above still works.

The Downside

There is an insidious down-side to all this. Best practice dictates keeping all those app folders – and especially the config files, rich with passwords and the like – out of the web root. In theory, we have that all handled with the .htaccess Deny on the _zf folder. But it’s a single point of failure. If it fails for some reason – say, the .htaccess file doesn’t get deployed – then we are fully exposed to a wide variety of potential mischief.

There is some additional obscurity you can obtain by changing the name of the _zf folder to something (practically) unguessable. I could even imagine a cron job that renames the folder and makes corresponding changes to the index.php file. But that’s probably excessive. And, in any case, as we should all well know, relying solely on security through obscurity is a poor approach.

But maybe this approach is useful to get an app off the ground with a customer who is stuck with his shared hosting package. At some point in the future, when the app is sufficiently profitable, maybe you can convince the customer to move to virtual private server hosting, at which point it’s a small modification to return to the standard ZF-recommend file structure.

Whaddya think?

Debugging PHP – Netbeans Code Templates for Zend_Debug and FirePHP

| Comments

I recently changed my day-to-day PHP IDE to Netbeans and it’s been a pleasure. [Don’t ask me what I was using before, too embarrassed to say.]

idea light bulb

In particular, one of Netbeans’ nice features is Code Templates, little snippets of text that you can insert into your document simply by typing a short prefix and then hitting the Tab key. They are actually smarter than just static snippets in that they can contain variables that are dynamically filled/changed as you edit. Hugely useful. I use them constantly for a variety of tasks, but my most common use of code templates is for debugging.

One useful debug tool is FirePHP in conjunction with Firebug, especially for applications using AJAX.

Several weeks ago, Jeremy Kendall (@JeremyKendall) helpfully pointed me to Zend_Debug::dump() which behaves like an enhanced var_dump(). I was actually gratified to see that it resembles one of my own hacks that I have been using for years, though Zend_Debug::dump() is, not surprisingly, more polished.

Following on that, Markus (@derstahlstift) posted a Netbeans code template that uses Zend_Debug::dump().

I admit that they are both cool, but I prefer to identify precisely where I am doing the dump, using PHP’s magic constants __FILE__ and __LINE__. This allows me to easily chase down the dump statements and remove them once the debugging is finished.

So, here are my own versions, both in the form of Netbeans Code Templates.

For FirePHP::send():

FB::send("${VAR default="variable"} = " . $$${VAR} . ":: ${func default="func"} :: " . __FILE__ . "(" . __LINE__ . ")");
${cursor}

For Zend_Debug::dump():

Zend_Debug::dump($$${VARIABLE variableFromNextAssignmentName default="variable"}, "${VARIABLE variableFromNextAssignmentName}", true); 
echo "

" . __FILE__ . "(" . __LINE__ . ") :: ${function default="function"} :: ${message default="message"}

"; ${selection}${cursor}

Whaddya think? Do you have any tricks/shortcuts that you use for debugging? Let me know.

Cheers!

Zend Framework View Helper to Lowercase Titles

| Comments

When working on a web app, I often find myself with my local development version open in one browser window with the live version open in another. To allow me quickly distinguish one from the other, I like to tweak the rendering in the development version by converting the title to lower case.

Since I have recently begun using Zend Framework for much of my development, I wrote a little view helper to handle it.

class App_View_Helper_HeadTitle extends Zend_View_Helper_HeadTitle
{
    public function headTitle($title = null, $setType = Zend_View_Helper_Placeholder_Container_Abstract::APPEND)
    {
        if (null !== $title && APPLICATION_ENV == 'development'){
            $title = strtolower($title);
        }
        return parent::headTitle($title, $setType);
    }
}

To use it, just add the helper path to your bootstrap. For example,you could do it in your config/application.ini with:

resources.view[] =
resources.view.helpers.App_View_Helper_ = "App/View/Helper"

The calls to $this->headTitle() in your view scripts or layout scripts remain unchanged.

As a side note, a great repository of Zend Framework snippets is at the aptly named site ZFSnippets.com.

Do you use any similar tricks or snippets in your development? Let me know via the comments.

Cheers!

Upgrading MySQL – Error 1045 in Configure Instance Wizard

| Comments

I recently upgraded my local database server from MySQL 4.1 to MySQL 5.1 on Windows XP. Got stuck in a jam involving root passwords and MySQL error 1045 that took hours to chase and fix. As a way for me to remember, and in the hope that it helps someone, else in a similar fix, I want to detail what happened and the silver bullet that did the job.

Separating Pingbacks and Trackbacks From Comments in WordPress

| Comments

I quite like my current WordPress theme: Fresh from Wolfgang Bartelme.

But now that I am getting some trackbacks and pingbacks, I notice that the comment handling is missing something that I generally value in a blog. When listing comments, I prefer the real, human comments to be separated from the pingbacks/trackbacks. Although the pingbacks/trackbacks are completely important for their SEO value, I find that they can interrupt the flow of the conversation, disrupting the continuity. In my opinion, it’s better to list them separately.

Fortunately, it’s pretty easy to do. The theme file comments.php usually contains a loop that renders all the comments. The core of the technique is to use the WordPress function get_comment_type() to determine the type of comment, then build two distinct buffers, one for which get_comment_type() returns ‘comment’, another for all the others (the trackbacks and pingbacks).

More details – though using a slightly different overall approach – can be found at Ryan J. Parker’s blog which, ironically, uses a modified version of the Fresh theme and where I suspect that this very post itself will appear as a pingback, nicely separated from the real human comments. ;-)

Conflict Between Google Translate Widget and Firefox Flashblock Extension

| Comments

The Google Translate widget allows webmasters to add on-demand translation of a website page. Very easy to configure and deploy.

The rendered widget appears in the browser as a simple select dropdown with options for all the language supported by Google Translate. When the user selects his desired target language, the widget is supposed to contact the Google Translate mother ship, translate the text on the page, and then add an fixed iframe panel at the top of the browser viewport, followed by the translation of the page. The translated page even implements onmouseover handlers for text-based elements that display the original source text. Sweet.

It has worked great for me in the past. But I recently did a new deployment and I was unable to get the widget to work. When I (as the user) selected my target language, I got the following:

Error: The server could not complete your request. Try again later.

A bit more poking showed that it was only happening in Firefox. All my other browsers were ok. Eventually, I narrowed it down to a conflict with the Flashblock extension. Disabling the extension solved the problem.

Now, the tough choice is to run without the Flashblock or without the Google Translate. But at least I can deploy for the customer.

2010-02-20 Update: The mere presence of the enabled Flashblock extension when visiting a page with the translate widget does not cause the problem. The issue only occurs when the page has Flash content and the extension is configured to block Flash on that page.

The reason: The widget appears to actually use Flash!

In the file

<a href="http://translate.googleapis.com/translate_static/js/element/main.js">http://translate.googleapis.com/translate_static/js/element/main.js</a>

the function wf() seems to add Flash embed code into the page. I imagine that Flashblock is detecting this attempted insertion and is doing its magic.

I actually find it interesting – and impressive – that the Flashblock extension is smart enough to not only find/block Flash content on initial page load, but also at any time after that. I imagine it monitors the DOM and is vigilantly swaps out any Flash embeddings with its own replacement button.

I can see that the widget creates several iframes, at least one of which has Flash content pulled from the domain translate.googleapis.com. It stands to reason – well, at least to me – that enabling Flash content for this domain should do the trick. But so far, no luck. ;–(

InfusionSoft API Support via Helpstream

| Comments

Man, InfusionSoft sure is not making things easy.

One of my customers recently moved to InfusionSoft for their CRM platform. They have a mailing that gets sent every morning to a list of opt-in subscribers. Prior to their move to InfusionSoft, we maintained our own database of subscribers. Now, we need to pull the recipients from the InfusionSoft database.

Fortunately, Infusionsoft offers an API to their data. It’s not the best API (as noted in Jon Gales scathing blog post), but it appears adequate, at least for this task. It’s an XML-RPC API, so the calls don’t technically require a dedicated client library, but InfusionSoft has helpfully developed one.

But it was a bit of a muddle trying to determine the current version of their released library and where to actually download it.

So I sent them a support request. And here’s the tricky part. Logged-in to the InfusionSoft admin interface, I see the following:

infusionsoft1

I click a link for “Help > View My Support Cases” and get sent to the “Fusebox”, whatever that is, where I am still logged in:

infusionsoft2

I click on the “Question” link and enter my question, complete with all detail that would probably be required by the Customer Support rep: account name, email, even a complete email signature, containing email address, snail-mail address, tel, etc.

I get my answer back pretty quickly. The answer itself is quite satisfactory, complete with the code I need to perform my little task. My application development proceeds in a straightforward manner, the API works as advertised. Kind of a pain to get there, but, for the most part, all cool in the end.

Or so I thought.

A few days later, I get an email message from the InfusionSoft system, the content of which is a comment from some random stranger, noting that he has the same problem as I have.

WTF? How did this totally random guy get access to my support request?

A bit of poking around and the truth begins to emerge from the fog: A Question is not the same as a Case. The latter is a customer support ticket, a confidential communication to the Customer Service folks at InfusionSoft. In contrast, a Question is apparently nothing more than an initial post on a forum thread. As such, a Question is completely public, exposed to essentially anyone (well, to anyone in the InfusionSoft community). Email, name, tel, InfusionSoft account name. Man, it’s just sheer luck that I hadn’t included my freakin’ password in my post.

Yikes!

So, I am now chasing InfusionSoft to get my post removed, or at least edited. We’ll see how that goes.

In hindsight, I can see that there is a link to “Cases” back on the main Fusebox page – it’s there in the screenshot above – which is undoubtedly what I should have used. And had I been more familiar with the Fusebox, I might have realized its dual nature, as a Forum/Community/KnowledgeBase (the Questions) on the one hand, combined with a confidential Ticket/Case system on the other, all built on the Helpstream platform.

But at the time, I totally missed it. It just never occurred to me that the Customer Support pathway I was pursuing, especially from a logged-in interface, would lead to posting in a public forum.

Would you have seen it? Let me know in the comments.

Cheers. And, let’s be careful out there.

2009-12-28, 22:00 – Update: Got a reply from the community manager, informing me that he has removed my post, per my request. He even sent along the text of my request, in case I did not have a copy of it. Very prompt, very satisfactory. All cool.

Oracle to Control MySQL? Very Bad Idea.

| Comments

“We are using MySQL, help save it”

The blog post above from Monty Widenius, the creator of MySQL, starts out this way:

I, Michael “Monty” Widenius, the creator of MySQL, is asking you urgently to help save MySQL from Oracle’s clutches. Without your immediate help Oracle might get to own MySQL any day now. By writing to the European Commission (EC) you can support this cause and help secure the future development of the product MySQL as an Open Source project.

And continues to explain the fairly obvious: that Oracle benefits more from killing MySQL than it does from helping it.

Anyone who uses MySQL should read Monty’s post and take at least some of the actions he suggests.