PipeJax - the Yahoo Pipes to AJAX bridge
UPDATE! Thanks to kentbrewster.com for showing me the Yahoo Pipes callback feature. With this I was able to write the php mirror file out of the system and have it work purely in js. Please see my latest post with the pure js implementation.
http://comments.deasil.com/2007/02/19/pipejax-pure-javascript-version-yahoo-pipes-to-ajax-bridge/
Ok, so I took some moments to rewrite what I had been using to build the “who else is writing about this” links into something more generic. It now let’s you use any Yahoo Pipe with any user supplied fields and automatically places them wherever you want in whatever format you want on your web page, more or less.
There’s a few steps involved in getting this to work for you, but they are all pretty easy, so bear with me. Here’s the instructions:
Prep Work:
- Download this tar file, it contains pipe.php, pipes.js and test.html
- get jquery.js from jquery.com - here is a direct link to the download.
- rename jquery-latest.pack.js to jquery.js
Installation:
- add pipe.php into your document root. You need to note down the url to access it in your browser.
- modify pipes.js - In that file you will see a line that looks like this:
- $.getJSON( “/wp-content/pipe.php”,
- change the part that says “/wp-content/pipe.php” to whatever the url you used for Installation Step 1 above. Something like this:
- $.getJSON( “/new/path/to/pipe.php”,
- add the newly modified pipes.js to your document root. You need to note down the url to access it in your browser.
- add jquery.js to your document root. You need to note down the url to access it in your browser.
- in your header (or header template) somewhere in the head block add these two lines (make sure that the paths to these two files are the ones you noted down in Installation Steps 3 and 4 - not /path/to/jquery.js):
- <script type=”text/javascript” src=”/path/to/jquery.js”></script>
- <script type=”text/javascript” src=”/path/to/similar.js”></script>
That’s it for installation. Now to actually use it you need to put some html in your file. If you look in the contents of test.html you’ll see that it has in the head block the two lines from Installation Step 5 (it’s set to work with my setup - so this file will not work for you unmodified).
The easiest way to see this working is to put it in your document root after updating the two script tags so that it points to jquery.js and pipes.js as they exist on your server. Once there, you should see (after a few moments while it loads the data) two sets of links appear. First should be 2 links that relate (at least a little) to snowboarding and then 4 links that relate to jquery.
If you look at the source code you’ll see several things. The key things are in the surrounding div tags. First, each of the two surrounding div tags have a class of deasil_pipe, this is the trigger that says this is something we need to pull links for. You can leave that as is. The second thing is you’ll see a deasil_pipe attribute, in this case it says deasil_pipe=’AHaR2Y242xGwcHpiZFUMqA’ - this is the id of the pipe you are trying to access, in this case it is my technorati search pipe. You can update that so that this will reference whichever pipe you like.
Then you’ll see a couple attributes that look like deasilx_[something], in this case it’s deasilx_text and deasilx_num - text and num are the names of the two user input fields I created for the technorati search pipe. By placing these deasilx_ attributes in the div, I am going to send their values to the yahoo pipe. So, as you can see, whatever user inputs are required can simply be added to this template. You can see in the first set I send deasilx_text=”snowboarding” and deasilx_num=”2″ - so that’s why you see two links about snowboarding. In the seccond deasilx_text=”jquery” and deasilx_num=”4″ - 4 links about jquery. Easy! Right?
So, that lets you set which pipe you want to access and what fields you want to send it. Then within each surrounding div (the ones with class=”deasil_pipe”) you see some more html. Right now there’s only three magic things here - things with class deasil_pipe_item, deasil_pipe_link and deasil_pipe_title. Everything else that is text or html w/out those classes will show up untouched.
deasil_pipe_item is how we should display 1 link. So for each element that the pipe returns, this html will show up once. An “a” tag with class of deasil_pipe_link will get it’s href set to that link and a span class with class deasil_pipe_title will show up with the title within the span. All of these things can be styled as anything else, so you could give the span another class or surround it with “b” tags to make it bold, etc..
One last thing, you’ll notice that on the surrounding divs (class=”deasil_pipe”) I have set the style to display: none. This hides the whole block until we get data back from yahoo (which can be flaky at times) and so the html will only show up if we get good data back.
That’s it - it may sound like a lot of work, but it’s actually fairly easy to work with, once you get the hang of it. Please let me know if you run into any trouble or if you successfully install it somewhere!








September 6th, 2007 at 6:07 pm
hi,
thanks for quality posts! found your blog (and this entry) looking around for ways to tweak the yahoo! pipes… very specifically, i’m looking for a way to integrate string searches of entire pages (url’s, you know, normal pages) - not just RSS feeds. There is no functionality for this in the pipes themselves (as far as i can see - and i’ve looked real hard), but it seems to me to write some ajax might be the trick?
would you have any idea or comment on where i could even begin to work around this? what specifically i would like to do is to search for keywords in the respective articles that the RSS feed entries link to. that means gathering a rss feed, and probing each “link” url for a certain string, and returning simply whether the string was found or not.
not sure if im making myself v clear here tho… but since you obviously have a pretty good hang of it, i thought maybe you’d have some insights to share.
keep up the good work,
knut.
(email: theosloexperience [howdy!] gmail.com i’ll check back here tho, of course… )
September 7th, 2007 at 4:19 pm
Knut, if I understand what you’re asking you want to take an rss feed, from which you will extract some sort of search term. Then you want to use that search term and search through specific urls that you have to see if that appears?
If that’s what you are looking to do, as you suggest there’s no way to do this strictly within the bounds of pipes. But the good folks at Brickhouse have developed a web services module that sends out the rss from pipes to a url you specify which can do the searching you are looking for and then return rss/json back to pipes to finish it’s way.
Is that what you were looking to do? If not let me know and I’ll try and suggest something else. :)
September 9th, 2007 at 7:38 pm
hi felix, and thanks so much for your response.
i’ve checked out the web service, and i suppose it could be what would do the trick. what i want to do is slightly different from what you’ve had understood (my bad completely), so i figure i’d put it straight, in case there is another solution.
what i want to do is to use predefined search terms and search through specific websites (ie. the articles from certain websites which are fed through their respective rss-feeds) and return simply whether the search term was found in the articles or not. for example, let’s say i would like to know if the string ‘police’ is contained in any of the articles of, say, three different newspapers today. i figured i could list three rss-feeds from these websites, not search the feed description itself, but the actual articles which the feed links to, by picking up the links (to articles) in the feed and doing the search on these url’s. i would like to return simply how many occurrences the string has - ie. not simple string occurences, but how many articles the string occurs in (once or more).
like i said, i guess the web service module could do the trick, but there might be an even simpler solution? i don’t know how i’d write the java script for it, however - would it have to be done ‘manually,’ ie. a script that actually goes out and does the search itself, or do you know if there are solutions made already that could be called upon to do this?
again, thank you so much for your response and concern. hope to crack this problem wide open soon, and your help is greatly appreciated.
knut