Getting cross-site content with JavaScript is cool, since it usually requires a tiny bit of extra effort when designing the feeds or deploying a callback proxy. Thanks to Yahoo Pipes it has now become very easy to get any content, even a whole web page and crunch it in your JavaScript web app.
First go to pipes.yahoo.com and Create a new Pipe. You’re going to have to log in.
Drag “URL Input” from “User inputs” to your pipe diagram. Now get back to “Sources”. If you’re planning on fetching a website, drag “Fetch Page”. If it’s a JSON or XML feed “Fetch Data” (all data will be automatically transformed ).
The next step is linking the elements together. Drag a link from the “URL Input” into you source’s URL Attribute and from sources output to Pipe Output.
Your pipe should look something like this:

Click “Save” and “Run Pipe…”. You’ll be taken to the pipe information page, where you can enter the URL which you want to proxy. Submit with “Run Pipe”, after it has loaded, select “Get as JSON”.
You should see some nice JSON with the transformed content. The URL to such pipe looks like this:
http://pipes.yahoo.com/pipes/pipe.run?_id=d53d0d4793aa292d3e02885ba9b22cba&_render=json&fromurl=http://stawecki.com
Now guess how to add a callback 😉
http://pipes.yahoo.com/pipes/pipe.run?_id=d53d0d4793aa292d3e02885ba9b22cba&_render=json&fromurl=http://stawecki.com&_callback=test
Surprise 😉 There is an underscore before “callback” attribute name. (weirrrd) For anyone who is new to XSS, “callback” is basically the name of a JavaScript function that receives the content. To show how it works, here’s a simple script that downloads a website and returns the response’s character length:


	function test(response) {
		try {
			alert( 'Content size: '+
			response.value.items[0].content.length );
			} catch(err) { alert('Invalid response'+err); }
		}

	function getURL(urlstr) {
		var ka = document.createElement('script');
		ka.type = 'text/javascript';
		ka.src =
'http://pipes.yahoo.com/pipes/pipe.run?_id=d53d0d4793aa292d3e02885ba9b22cba&_render=json&fromurl='+
		escape(urlstr) + '&_callback=test';
		var ks = document.getElementsByTagName('script')[0];
		ks.parentNode.insertBefore(ka, ks);
	}

And here’s a live version to try:
Simple Pipes Callback

Here’s a slightly more complicated example. I’m getting HTML content, parsing it internally and listing all links on the website (NICE!):
Link Scraper Demo

These examples used “Fetch Page” in pipes. Here’s an example of reading an RSS feed using “Fetch Data”. This script alerts the most recent news item from an external RSS feed. Notice that Yahoo Pipes conveniently transforms any feed to JSON (or XML if you really really want).


	function test(response) {
		try {
			var newsItem = response.value.items[0].channel.item[0];
			alert( newsItem.pubDate +" - " + newsItem.title +" - " + newsItem.description );
			} catch(err) { alert('Invalid response'+err); }
	}

	function getURL(urlstr) {
		var ka = document.createElement('script');
		ka.type = 'text/javascript';
		ka.src =
		'http://pipes.yahoo.com/pipes/pipe.run?_id=4d625dfe6977e71acb45db4aa51726a6&_render=json&feedurl='+
		escape(urlstr) + '&_callback=test';
		var ks = document.getElementsByTagName('script')[0];
		ks.parentNode.insertBefore(ka, ks);
	}

And here’s a live version to try with BBC News RSS feed:
Simple RSS Callback

Now go and cross site the hell out of the internetz! Soon slightly more practical examples.