Year: 2015

Having read my previous post about how I found myself messing around with visual programming as an online marketer, you might have wondered: and what would be the everyday uses of those scripts when dealing with ad campaigns and web sites? Well, let me share some examples from the last few years I was working with these web automation tools to illustrate this:

Overcoming limitations of AdWords: finding more manual display network placements

Have you ever wondered whether AdWords will suggest you all the relevant display network placements, Youtube videos or Youtube playlists when you try to add them by entering relevant keywords? Well, the answer is that relying only on the ad management interface of AdWords, you will miss a lot of relevant placements. Fortunately, with some web automation skills, you can quickly build a script which finds even more relevant placements for instance by executing site searches on Youtube for a certain list of keywords, automatically pressing the next-next buttons and generating a simple list of URLs based on what has been displayed on the search result pages.

Analyzing your data the way you want: exporting external link data in a meaningful format

Although Google Webmaster Tools (a.k.a. Google Search Console) lets you browse through a huge list of web pages where a certain site of yours is linked, you cannot really export that data in a usable format, such as linking domain, linking web page, linked page in the same row. Although you could click through the list of linking domains, then the list of linking pages and export a bunch of tables based on this hierarchy, this sounds like a kind of repetitive task which can be fairly easily automated. Adding a few more steps like scraping the title of the linking page plus the anchor of the link, you can end up having a really informative list of your external links – at least of those which are displayed by Google.

Analyzing your data the way you want: obtain raw engagement data

While Facebook shows you some insights about how your pages or posts are performing, you cannot simply grab the raw data of these statistics: such as the number of visitors liked or shared certain posts in a given timeframe. But you can build a script which automatically scrolls and scrolls and scrolls – and extracts any data about the posts displayed. Having all the data in a spreadsheet format, you can visualize it the way you want. As a bonus: you can even do this with your competitors’ Facebook pages.

Automating repetitive tasks: checking link building results

Way back when we have been building tens of thousands of links on web directory sites, no link submission software could provide us with detailed and reliable data about which directories had accepted and published our link submissions and which had not. Without knowing how many links were eventually generated and where were those links located, we could not create detailed reports for our clients. On the other hand, the biggest problem of directory link building was that  you never knew at the time of submission where the submitted link would be displayed in the directory, so the challenge was not only going through a list of URLs and see whether our link was found on those pages or not, but you had to look through the entire directory to figure out where exactly that link was. All in all, this task was more complicated than curl or wget a list of URLs and grep the results. Before I knew how to automate this process with visual scripting, we had to do this highly repetitive task by hand – so scripting could save us a lot of manual work.

Process spreadsheet data: check and merge what’s common in two tables

When you have to work with email lists and related data coming from different sources, you could quickly diff or merge two .csv files with Unix command line tools. But sometimes not everyone in your organization possesses those “geeky” skills to fire up awk for that, and many times you are also too lazy to find the best solution on StackOverflow. In these cases, with automation software, you can even create an .exe file with an easy to use interface where two files and a few more parameters are asked, such as which column’s data should be matched in the other spreadsheet to merge a table with the unified rows based on those matches – or whatever you can achieve with regular expressions, if / then statements and loops.

Extracting structured data from unstructured source: list of products in a website

Unfortunately, still there are many web shop owners who are running their sites based on proprietary webs hop management systems, which are not prepared for simply exporting the list of products from, or not in the appropriate format, with all the desired data, etc. In these cases, it is very handy if you can quickly build a script which scrapes the entire webs hop and outputs a spreadsheet of every product, containing all the important attributes and product data. Based on the result, you can start working on either the on-site SEO or importing those lists to Google AdWords of Facebook Ads.

Migrating web sites: exporting and importing from/to any CMS

There are quite a few ways of importing data into WordPress, but still, you might miss some features which can be normally accessed only if you upload the content manually, such as attaching images to a certain post or set the featured images. Not to mention that before that, you’ll have to get to the point of already having the data extracted from the old web site to a structured format such as XML and CSV. As many older CMSes and proprietary content management systems do not have such data exporting features, this part of the job could be also quite complicated, if not impossible. On the other hand, with some web automation skills you can extract any data in any format from the original site and imitate a human being filling out the corresponding data simply automating the new site’s administration interface – you don’t have to rely on any export-import plugin’s features – and shortcomings.

Web spamming: black hat SEO, fake Facebook accounts…

The tools I’m using for automating the above tasks are originally meant for creating accounts, posting content to a wide range of sites: thus spamming the entire web — but this is something I have never used these tools for – believe it or not 🙂

Coming next:


No matter whether you are planning to create a new landing page or an entire web site, these websites/collections might be useful sources of inspiration, showing the latest and greatest in web design and data visualization.

Themeforest / latest WordPress themes

WordPress is a leading light in contemporary web design/web site building, and Themeforest provides the biggest collection of premium templates for it.

Since 2011, when I registered on this site, I have spent countless hours browsing its Wordpress theme collections, looking for nice stuff for both clients’ sites and for my own projects. …and sometimes just for the sake of seeing something new, nice and interesting. I must admit this is generally quite a conservative collection of web design ideas, as these templates were made for being sold to as many buyers as possible. This is the reason why the popular themes are many times adhering too much to a kind of standard taste, and this is why it takes some time to find really cutting edge designs or novel ideas among all of these ordinary themes. Some tips for finding the most inspiring designs faster at ThemeForest:

  • Start with the list of latest WordPress themes. The list of most popular themes might be interesting only if you are a newcomer, or not dealing with web development on a daily basis.
  • Restrict your searches with ”buzzwords”, that is keywords reflecting the latest trends and techniques, like masonry, isotope, megamenu, infograph, whatever. Keep in mind that as these words become mainstream, so they will become quite useless. Think about parallax (950 templates) or even the keyword: ”responsible” (more than 4000 templates).
  • Unfortunately, designers are generally concentrating on portfolio sites, fancy home pages, and one-page templates, so it is quite hard to find web designs with interesting ideas about organizing content and showcasing data. This is why whenever I open a template demo site, almost the first thing I do is to check out the Blog section and see what happens if the nice pictures included in the demo data set are not influencing that much the overall look and feel of the template.
  • If you have found a nice template, it’s usually worth checking out the other templates of the same developer too, they might have some more inspiring web design ideas showcased in their portfolio.

Some WP templates I really like

I discovered these templates a few years ago, and I think they have stood the test of time (and most importantly they are still sold at ThemeForest):

This template is rather about information than design, featuring a big search form as the centerpiece of this template:

And finally an other minimalistic theme never sold on Themeforest – this is just an evidence of the fact that there can be found inspiring template designs outside ThemeForest too (although they are much harder to be found on the interweb): / jQuery plugins

From a jQuery plugin which makes your browser fart when scrolling (NSFW), to algorithms generating nice backgrounds like the one you can see as the featured image of this posts, a wide range of different jQuery plugins are aggregated to – a tidy repository of jquery plugins. There are almost one thousand items listed on this site, like these ones which showcase:

… and many more jQuery plugins and functions to enrich your custom web designs or even create an entire landing page with. There are also a few better-known plugins gathered at this site, such as Isotope or Lettering.js, but the majority of the plugins are quite experimental – and therefore quite inspiring too.

Similarly to ThemeForest, Unheap also provides invaluable help with its easy-to-use interface to browse through the lists of many different plugins which have been gathered manually and listed in a uniform way. The demo links lead directly to the developers’ web page, and similarly to ThemeForest portfolios, it is also inspirational to browse through and see the other projects of these jQuery developers.

D3 Gallery

…or everything that is made with D3.js library, or made by Mike Bostock, a key developer of D3.js data visualization library, and also a former leader of complex data visualization projects and interactive storytelling articles at New York Times:

It is more about data visualization than web design, but for every web developer or online marketer who deals with visualizing information, the above examples can be very inspirational too. Not only when it is about building a web site but also when the data, we are working with, have to be presented in an outstandingly new way.

There might be a lot of other web sites showcasing the latest trends in mobile and web interactions. It’s sometimes just too hard to come across them, so I am quite sure that there are a ton of web sites out there I should be aware of.

It’s not so well known but there is a place where Google displays what it thinks about you – or at least what it is willing to tell you about it. If you think that Google knows everything, then you might want to visit this web page and check it out:

As for what Google has registered about me, the results were very-very poor: I had to uncheck 30 interests out of the 39 different interests Google has previously attributed to my Google Account. I must admit that as an online marketer perhaps I am not the easiest to profile since I am visiting a lot of different web pages, doing a wide variety of search queries, watching a lot of videos and ads, but still the original list was full of irrelevant interests like Dance & Electronic Music or TV Reality Shows or Billiards – topics I have never sought for.

As a frequent user of Google’s services like AdWords or Analytics, I tend to think that there are a whole lot of things Google knows about us statistically and builds many services upon that (such as conversion optimizer or similar audiences) without disclosing too much about its knowledge to us.

Perhaps I have overestimated Google’s ability to identify and categorize their users based on their activities? Once I was told by a Googler that cookie is the new keyword, but if there are so many irrelevant pieces of information attached to those cookies, there is still a long way to go.

At the age of twelve

My mother bought my first computer at times when everyone having a computer used to learn to code as well. Although I had been enthusiastic about both coding and playing with my Commodore +4, I was never able to see through many lines of code and not to make silly mistakes while transforming my ideas into Basic programming language. So I never got immersed to coding and didn’t become a programmer eventually.

Getting back to programming with web automation

Way back in 2012 when I was already doing all sorts of search engine marketing I started to code again. I was looking for effective tools to automate my mass link building tasks, and instead of sticking with yet another link submission software, I came across a wonderful generic web automation tool. It took me two weeks to write – and other two weeks to re-write – my first script which could do the tedious work of checking the status of our directory submissions one by one. I was just happy that this script was virtually replacing a half-time employee, and wasn’t really thinking about how important visual programming would become for me in the future.

Visual programming for online marketers

During that last three years, I had been spending more and more time with using these tools – originally built for web spamming. This experience has gradually transformed what I think about an online marketer could and should do when it comes to aggregating, processing and analyzing data – from the web, our own analytics or other internal data sources. Although I admit there are faster and more scalable ways of doing this, having a few easy-to-use software at hand makes me able to relatively quickly get insights from the data I’m working with or complete those tasks which have a highly repetitive nature – without being a programmer or employing one.