Mateusz Stawecki

Thoughts on Mobile, Reactive & Serverless

Posts from the ‘Tech Log’ category

I’ve been recently having a lot of thoughts on live data and visualisation. Tweet Splash presents data available through Twitter API. This time, I’m going to set up cheap event tracking, which you can very easily use in many different situations and review later on Google Spreadsheets.

1. Setting up and playing with Google Docs: Forms

Log into your Google Account and go to Google Docs. Press “Create New” -> Form
I’ll call my form “Blog Searches”. Let’s have only one “question” which will represent our event or query made by a user. In my case, question title is “keyword”, question type Text and question is required. Click “Done” and make sure you delete “Sample question 2”. Save the form and click the link at the bottom to see the public version of your form.

It should look more or less like this. You can try, if it works by posting a test keyword, then go back to the document list and enter the form again, this time in a Spreadsheet form.

Now it’s time for the hacky part. Go to the public version of the form and look up the source (usually right-click “View Source” ;] ).

We need to retrieve 2 values: Form Action URL – to know where the form is posted to and Input Text’s Name, to know what’s the name of out “Keyword” parameter.

Form Action URL will be here (in red, it’s the attribute value of “action” in the “form” tag):

<form action="https://spreadsheets.google.com/formResponse?formkey=dDVCMlYzMmpuWTg0VUhZUkltaXl5OEE6MQ&amp;ifq" method="POST" id="ss-form">

Input Text’s Name (in red, attribute value of “name” in the only “text” input field):

<input type="text" name="entry.0.single" value="" class="ss-q-short" id="entry_0">

Now let’s try a small trick I discovered. Usually the form data is sent through a POST request, however turns out Google doesn’t mind, if you send values over a typical GET, which means you can put them as a part of your URL! In my case I’m going to this URL in my browser (note that I ignored the “ifq” parameter):

https://spreadsheets.google.com/formResponse?formkey=dDVCMlYzMmpuWTg0VUhZUkltaXl
5OEE6MQ&entry.0.single=Testing+GET+form+submission

Basically, to the URL, you append an “ampersand symbol”, name of the field e.g. “entry.0.single”, an “equals sign” and finally a message. Remember that a text in a URL has to be properly encoded e.g. spaces are turned into a “+” or “%20”, a browser will do it for you.
If you check your spreadsheet now, you should see the message:

Now I will show two methods for mixing this up with your website or service: sever side and client side.

2a. Submitting an event – Server side

I’m going to show the server side method first, since it’s really the best way to do it. If you can you should avoid exposing tricks like this, primarily because it’s easy to recover your form URL from a piece of a publicly available JavaScript, which means somebody might play a nasty trick on you and post some trash into your spreadsheet. On the other hand, if you do it through JavaScript, the user can decide not to be tracked by turning JavaScript off. Here are some examples of a server-side post.

In PHP there are several ways of fetching a URL, this is an example of posting a search query made by user. You can put this code in your index.php and it won’t be called unless parameter “s” is passed. Change it to the parameter name you are using for search or page name, or put a different version of the fetch in different files and provide a predefined event description string.

if (isset($_GET['s']))
file_get_contents(
'https://spreadsheets.google.com/formResponse?formkey=dDVCMlYzMmpuWTg0VUhZUkltaXl5OEE6MQ'.
'&entry.0.single=' . urlencode($_GET['s']) );

Java: (works also on App Engine)


public class ExampleServlet extends HttpServlet {

	public void doGet(HttpServletRequest request, HttpServletResponse resp)
			throws IOException {
		resp.setContentType("text/plain");
		if (request.getParameter("search") != null &&
			!request.getParameter("search").trim().equals("") )
			ExampleServlet.touchURL(
		"https://spreadsheets.google.com/formResponse?formkey="+
		"dDVCMlYzMmpuWTg0VUhZUkltaXl5OEE6MQ&entry.0.single="+
		URLEncoder.encode( request.getParameter("search") , "UTF-8") );
	}

	   public static void touchURL(String address) {
	    	  try {
	              URL url = new URL(address);
	              url.openConnection().getInputStream().close();
	          } catch (Exception e) {
	        	  e.printStackTrace();
	          }
	    }
      }

2b. Submitting an event – Client side

If you’re not lucky enough to be able to modify your backend code e.g. your using Shopify or similar, but you have access to the site’s templates and you can put in some JavaScript, you can still submit an event. Here’s how you do it in JS:


(function() {
var paramsplit = location.href.split('s=');
if (paramsplit.length > 1) {
var keyword = paramsplit[1].split('&')[0];
var ka = document.createElement('script'); ka.type = 'text/javascript';
ka.src =
'https://spreadsheets.google.com/formResponse?formkey='+
'dDVCMlYzMmpuWTg0VUhZUkltaXl5OEE6MQ&entry.0.single='
+ keyword;
var ks = document.getElementsByTagName('script')[0];
ks.parentNode.insertBefore(ka, ks);
}
})();

The script above performs a string split looking for parameter “s=”, which in my case is the search query parameter. If there is none, there will only be one element in the array – the location url itself and the rest of the script will not be executed. Otherwise, another split will separate the keyword from other parameters and the URL to the spreadsheet form will be fetched like an external script file.
This method has some pitfalls. First of all, anyone can see this code and start sending some trash or fake data. It’s also not very clean, because it causes a JavasScript parse error. The browser basically tries to execute the fetched HTML Form code as JavaScript. In order to avoid this you can deploy a simple Google App Engine application or a PHP script somewhere that fetches the URL but ignores the content and provide the URL to this application instead of the Form URL. You can use the Java example from before.
Update: you can also retrieve it as a valid response by calling the form through Yahoo Pipes! Check out my next article on how to do that. Great if you’re expecting a lot of form submissions but don’t want to host it 😉
To hide your code you can try tools like http://www.javascriptobfuscator.com/ , but currently there isn’t a 100% secure method of doing this, that’s why server-side submission is the best option.

3. Reviewing collected data

Let’s get back to the spreadsheet now. Of course, we can just observe the events here just as they are, but there is just one more trick I will reveal 🙂
Go do a different cell e.g. D2 and try a sorting formula:
=Sort(A1:B1000, A1:A1000, FALSE)
This will sort the whole data set based only on the date. The last attribute is FALSE which means it will be descending sort. This way you don’t need to scroll down to see the newest events.
You can also publish the top of the sorted data set as an RSS feed, by going to the Share menu, “Publish as a web page”. Set automatic publishing (unfortunately far from real-time ;/ ) and generate a link in the appropriate format to your sorted data set in cells e.g. D3:E15. You can use tools like http://twitterfeed.com/ to link it to your Twitter notification account!

4. Use cases

I started off with a simple use case of event logging on websites, but there are many other ways in which you can use Forms. You can do any type of logging information: from servers, mobile devices, receiving errors with stack traces, receiving other server notifications, keeping a simple backup database. First off, a lot of you might say: this is ridiculous! you don’t use forms for suff like this! You use specialised analytics and loggers etc. That’s true, but still the beauty of Forms is it’s simplicity. You can very quickly deploy any kind of logging or tracking and see, if it’s even worth going further. So, before you spend hours implementing a cool fast real time notification system for your platform, see what kind of data you might get by taking Forms for a spin.
And that’s it for now. Next time i’ll show how to deploy a dedicated real-time event tracking application.

2 Comments

Name: Tweet Splash
Motto: Make a splash for your event, conference, presentation, gathering…

Link: http://tweetsplash.com/

Summary:
JavaScript based Twitter client for real time Twitter search, easily configured for any event. The purpose of this program is to quickly deliver a “splash screen” on a monitor or projector that will engage event participants into a discussion on twitter.
Tweet Splash is also a YouTube API, Google Mobilizer mashup.

Everything is embedded in one file, easy to customise, great for playing around with Twitter Search API.

Features:
– Link preview
– Remote YouTube jukebox
– Polls from tweets

Coming soon:
– URL Handler
– Html5 Local Storage for configuration

Leave a comment

This scenario is for people who want to put their Macs to sleep after a certain activity that you know will take ‘x’ amount of time, like: watching a movie, downloading a file.

In other words: auto shutdown Mac after certain amount of time.

There’s quite a few apps that might do this for you, but I’m going to show you how to do it the geeky way using the Terminal 🙂

Let’s say you’re going to watch a 2h movie and you want your Mac to go to sleep right after it finishes. Open your Terminal and write:

sleep $[120*60]; osascript -e ‘tell application “System Events” to sleep’

The first part with ‘sleep’ calculates the number of seconds in 120 minutes using bash’s arithmetic expressions and passes it to the ‘sleep’ function. After ‘sleep’ wakes up, we run a simple script that puts your Mac to sleep. Of course you can put any amount of time you need (it’s usually better to put too much than too little 😉 )

If you plan on using this quite often, it’s best to write a short script:

#!/bin/bash
d=$1
while [ -z $d ]; do
read -p “Duration (minutes): ” d
done
sleep $[$d*60]
echo “. . . z z z Z Z Z”
osascript -e ‘tell application “System Events” to sleep’

Save it somewhere in your path, if you can (enabling and using “root” user in Mac OS X), remember to do a ‘chmod +x’ on the file and you’re ready to go.
If you don’t want to use “root”, put it somewhere in your user directory e.g. make a directory “scripts” in “Library” and put your script there.
Create a “.bash_login” script file that will append your private scripts directory to the current path:

#!/bin/bash
PATH=$PATH:/Users/mateusz/Library/scripts

If you’re ready, just pass the number of minutes as an argument (I called my script “sleepafter”):

bash-3.2$ sleepafter 120

The “while” loop in my script makes sure that “sleep” has a value to work with. This way you can easily run the script from Spotlight:

bash-3.2$ sleepafter
Duration (minutes): 60

If you are invoking a task that ends the application process after finishing, then simply add the “put to sleep” script at the end. E.g. when compiling something:

make; make install; sleepafter now

E.g. when using applications like ‘wget’ for downloading files:

wget “http://server/bigfile.bin&#8221;; sleepafter now

Let me know, if you have any good ideas on modifying that script!
That is all 🙂

Leave a comment

I’ve done a tiny bit of research to find nice templates, but the best one I found was by accident, while I was looking for something completely different on iPhone Dev Tips blog. The link was to a Russian redirection service, which took you to a PDF on Google Docs – look here, but if it’s not there anymore, I made a mirror just for you!

iPhone Design TemplatesDownload

My favorite. Simple, couple layouts, I think most interesting are the first one and the last one, which can work as a flow diagram. The original “single” iPhone wireframe picture can be found here.

Android Design TemplatesDownload

The funny thing about Android is that there so many different devices! Well, this PDF covers the most. Grab it while it’s hot. Source: neurosoftware.ro

Mostly Web, iPhone, iPad and many more freebies – See here

10 free printable design wireframe templates. I used to just grab a screenshot of Firefox on an empty page, print it and draw, but this is pretty neat.

Yet another iPhone Design Template – See here

Very nice looking, with extra space for notes, title, page numbers. Probably better, if you want to share your designs and want all the explanations there. Not the best, if you want to draw a lot of different views or you are experimenting – take the first template instead.

Have fun!

1 Comment

I couldn’t really find any nice RTS games for my android, so I though that it would be quite awesome to run StarCraft somehow. Of course remotely!  The only running PC emulator for android  is currently “aDOSbox”, which is unfortunately really slow 😦 so there’s no way of running the old first version of WarCraft yet. Anyway, here’s my idea.

What you need: Andoid Phone, androidVNC, preferable a GNU/Linux or BSD server with tightVNC and Wine.

You could just run a VNC server on any Windows/MacOSX machine, but what I was thinking about is having a server somewhere that will be able to run StarCraft when I want. I don’t need it at home, where I’ve got a full fledged computer.

Short instructions: install StarCraft with Wine on your Linux machine. You’re gonna have to configure tightVNC to 640×480 – 8 bit resolution, because the program won’t do it for you on a vnc X server (I’m assuming we’re not running proper X session on a server). To do it, find your “tightvncserver” script (e.g. “which tightvncserver”) and edit the variables. Then “wine starcraft.exe” and we’re off 😀

Here’s how it works for me:

Set your androidVNC to fit screen and touch -> mouse click. The screen is a bit crunched to fit the screen, but I think everything is fairly readable, if you know the game. The stretching is not smooth, but it looks alright. It’s good to remember the hot-keys  for most actions, even building, because trying to hit the small command buttons without a pointer or a stylus is a pain. I tested it in London over the Internet connected with Wi-Fi to my server in Warsaw, it was playable. I don’t even want to try over 3G 😛

I don’t really play games on my android, but it’s not what’s important here. It’s the Geeky Time Excellent! 🙂 Cheers!

Leave a comment

Last approach to giving J2ME some delightfulness

I was recently working on a really small set of applications for Java as a part of my University assignment. I managed to set up a simple PHP script on one of my servers to get GameTracker.com data in XML. I first did an ajax-based web service on tomcat, then
a swing application. Both using SAX parser, it worked great. The last step was a mobile application. Building a basic version was almost effortless, because I was using SAX, it was just a matter of generating a table from a set of data. It was nice to see something working, but it was ugly! I didn’t spend too much time on it, so it didn’t seem finished. With some spare time, I thought about taking care of the interface. I used to make some J2ME games, so I knew my way around a GameCanvas. Coding a new cool GUI took me
twice as long as coding the previous basic application, but at last it looked nice.

The mobile application itself is just a tech demo, who the hell needs a game tracking feed on a mobile! However, if I needed to develop a similar tool I’d think twice before using the standard GUI API. It’s just way too old. There are many interesting J2ME applications and in theory, there should be a good market for them, but have you heard anybody saying “Wow! I got this cool Java app on my phone?” – maybe once or twice in life. I’ve tried J2ME VNC by the way and it’s annoying. “J2ME app scene” consists of not so playable games and some strange looking apps that you can find on very dodgy J2ME software sites.

Oh well, can you blame a technology that is about 10 years old? Here’s a shot from MIDP for PalmOS

It’s PalmOS kids! The coolest toy to be around before Windows CE came out (and became useful). I loved the good old PalmOS 🙂 Ahhh… I guess, that’s the price for a “standard” compatibility. Especially the visual part of any J2ME application will be hard to control depending on the device. I mentioned quite popular Java Games, but I’ve seen many of them go wrong!

Well, seven years later we have iPhone, then Android. Making a nice interface on them using standard tool set is really easy. The devices are more powerful, more colors, better resolution. Mobile platforms are focused more and more on design and style, which is nice… apps are for people and if we use them daily it’s good to look at something nice and readable. Obviously it’s quite silly to even compare those platforms, but the general summary is: producing a nice looking app for iPhone or Android is effortless, while J2ME will always look like 90’s.

Enough of this random talk 😉 Here’s my try on adding some style to the great functionality that J2ME still has to offer.

So… which one is more Quake-ish 😉

1 Comment