I use Diaspora I recently contributed again to Diaspora to help fund the next phase of development. Diaspora feels right for me for the following reasons…

A customer not an audience

It has a clear funding model, it allows you to be a customer of service rather than an audience for advertising or a source of demographic data. This isn’t a minor thing, it is actually a unique feature. Whether it is sustainable or not will be seen. Will people value a social network in the way they do Wikipedia? My feeling is that certain people do and others might and that could be enough to fund the network for everyone.

It acknowledges the primacy of the user as the creator of content

The other social networks allow you to extract your content to some extent but Diaspora correctly puts the user and the content they create centrally and makes it straightforward to extract and use yourself. The ability to federate and even pull your content and publishing entirely under your control should you wish to clearly goes further than any provider today.

It returns control to the user

It allows you to put some measure of control back on your online social life. Although this has now gone more mainstream with things like Google’s Circles Diaspora was the first to properly implement it and go through the real-world feedback loop. Diaspora’s Aspects allow to segment your network by audience and interest. They are a surprisingly powerful tool.

Is this enough?

Diaspora may not succeed, network effects rely almost entirely on volume of users and therefore it is critical that Diaspora has just enough use that there is some kind of feedback loop and you do not feel like everything you are doing is just being fired off into a void. However it does not have to be as successful as Google+ or Facebook to succeed in providing a valuable service to those who have concerns about control and trust.

Now the key lookup may be a range of keys you are interested in but in reality there is nothing in Couch that is similar to the SQL “WHERE” clause.

So if you cannot do queries then how do you relate data? Well that’s the thing about storing documents instead of rows, if you have related data then you have to ask whether that data has any meaningful existence outside of its parent. In relational terms it is like asking whether you ever access the content of table outside of JOIN with its parent.

Initially you might think: of course I do! But often data is often explicitly related to its parent’s primary key by things like ORDER and GROUP BY. In these kind of cases then you move the related data into the parent record, effectively denormalising to avoid a lookup.

If the data does have a meaningful existence outside the parent (for example in Wazoku comments are an example of a piece of data that exists separately from the thing they are a comment on) then you have a few options but essentially instead of querying you are still trying to do a direct key lookup.

The first simple case is to include a reference to key of the related data in the associated document. Then from one key lookup you can go direct to the next. As an example we store a list of comment document ids on any document that can be commented on and then we can load the comments as needed (often the count of the comments can be as relevant as the full content). I describe the ids used this way as “forward references” as they lead you on to the related document.

The second, slightly more involved approach, is the creation of a view that allows the document to be looked up via an alternative key. For example if we store the document id of the thing being commented on in the comment document under the key comment_on we can then create a mapping view of all comment documents to their comment_on key. Then given any document we can simply do a direct lookup on the key in the view to determine whether it has any associated comments.

The final common technique I use is something I refer to as “unrolling” of collections. So again we create a CouchDB view that consists just of a map job and in it we take each item in an array of “forward references” (related document ids) and emit a document in each view mapping the id to the current document id.

So if an idea document has five comment forward references the resulting view will have five documents, each relating a comment document id to the idea document id.

If things get more complicated then I also have the Couch databases indexed in Elasticsearch and in Neo4J and these alternative views of the data give me powerful adhoc queries on properties or relationships in the data.

In general though I am always trying to think ahead as to how my documents relate and then express that in terms of a key lookup so that I am always working with the simplest case.

However I often forget how exactly it works so I’ve decided to write up my latest remembering of how to do this (via John Hartley’s help and reminders) so I have something to look up next time I forget.

The first thing is that the patch decorator takes a string that represents the fully qualified name of the stub/mock you want to create. In a Django app for example that means you should include the app name at the root. The name also reflects the local name of an imported item. Something I commonly do wrong is to bind to the absolute name, say ‘random.choice’ rather than ‘myapp.mymodule.random.choice’. If you are in the situation where your stub is correct when you call it directly but never happens when you run the code under test I am pretty sure that naming will be at the root of your problems 95% of the time.

For each string argument you have in patch you also need to define a parameter to the test function, this will contain the actual Mock object and is what you use to actually stub the value to what you want it to be for the test. Use names that make sense here, stub_db, fake_file_reader not just mock1, mock2 and so on.

With these relatively few reminders in place you should now be in a position to stub simply with Mock!

The basic setup is an app.py, requirements.txt and the epio.ini. Requirements obviously just has bottle.

Epio.ini

[wsgi]

entrypoint = app:app

requirements = requirements.txt

The app.py file should be:

import bottle

app = bottle.app()

@bottle.route('/')
def home():
  return {"message" : "Hello world"}

That should give you a basic JSON service running quickly.

{
  "query" : {
    "more_like_this" : {
      "like_text" : "testing",
      "min_term_freq" : 1,
      "min_doc_freq" : 1
    }
  }
}

So firstly, there’s a command-line client that you install via pip and you effectively deploy to the platform via a client-command, SSH keys and what looks like git on the server-side. This is more like Gondor than Heroku (which is intimately linked to git). It means you have your choice of source control and if you want to be a Python purist you never need to step outside of Python for everything you are doing.

Applications consist of essentially one configuration file that states where the WSGI application is and what the requirements file is. Compared to Gondor it is a very simple setup but it did feel that it could be even simpler if it made convention-based assumptions such as the requirements file being called requirements.txt, for example.

Leveraging WSGI and configuration this way gives a very flexible platform and I was able to get both Flask and Bottle to work (the former very quickly because it has documentation, the latter via trial and error that might require its own blog-post). I didn’t have time to try Django but I felt pretty confident that I could get whatever framework I wanted working once I understood the basic setup.

Unlike Heroku, Epio provides a fixed framework for executing the apps. It seems you will be running behind NGINX and Gunicorn. Both are good choices and I certainly like them but if you want to play around with different servers like Tornado or CherryPy you may prefer Heroku’s more open deployment model. I did like the way that you can use the configuration file to have NGINX serve static content directly.

Epio naturally has less of an ecosystem than Heroku but has Solr, Postgres and Redis out of the box. All solid choices and covering off the majority of what I would need. I was certainly grateful that I didn’t have to grapple with remote database administration and could prototype apps with just Redis.

Deployment and logging have kind of rough edges. Being able to access logs directly from the application page was a win for me, however when I was struggling to define the WSGI entrypoint correctly it seemed as if the application wasn’t being really compiled until the first request comes in. I would see an entry confirming a new deployment but then nothing until I hit the app. I think there should be some kind of sanity check of what you have uploaded to see whether it will even run.

Right now epio is providing a Python-based cloud deployment platform with a sensible set of supplementary services and low opinion about the source control system to you use. It feels like if this had been around at the start of the year it would have blown me away. However now there is more competition and therefore questions of price and ease of use will matter in terms of  how compelling it is to use the service.

If you do Python web development I would definitely recommend you sign up for beta and give it a go yourself as it seems a very solid prototyping platform. If you are not a Ruby and Git fan then you may well love what is on offer here because it is already very convenient, makes few demands on you and gets your web app public in minutes.

The first advantage is that Google Apps that are bound into your domain allow you to create something that is easy to access for an existing user (no additional login is required) but also gives you peace of mind that you are exposing virtually zero surface area for attack.

The second is that for Python at least it is easy to access a very full featured environment with a minimum of code. Want to send emails, have task queues, access to memcache, serve static content? It is all a YAML configuration line or import away.

I love services like Heroku but a lot of internal apps have relatively light usage and benefit from the batteries included approach rather than combining various plugins. It makes it easy to switch between different approaches and react to different demands.

The first time I saw it I was very disappointed. While I cannot knock its authenticity it is a profoundly depressing document. While there are the standard statements about passion and having the freedom to make what you should rather than what you are told to; there is much more about poverty, tiredness and scarcity.

If I read this I would say that working for startups is a mugs game. You’re far better coming in during the expansion phase when salaries are higher and the business case better proven.

The many references to tiredness and lack of sleep is also revealing. What I have discovered is that tremendous pressure is put on you to deliver product in a technology startup and this should be resisted at all costs. Sustainable pace is more important in small organisations than in large ones. In a large organisation you can actually burn out a team to achieve a goal because you probably have access to the resources to replace them. In a small one, once you’ve wrecked a team (probably including yourself) you have no way of replacing them and a death spiral will inevitably set in as decision making becomes progressively worse. Remember that a startup should aim to deliver progress not product. Don’t work with people who don’t understand this.

Money, frankly seems to be the missing ingredient from this list of reasons. Maybe adding “because late-stage equity options are worthless” would ruin the overall tone. Many people, especially investors, are involved in startups because they offer potential massive returns in a low growth environment. Americans are much more open and brash (you might even say vulgar) about this with talk of flipping and sale valuations of millions of dollars (often farcical as in the case of Groupon who merely had the bad luck to be caught before their IPO).

Even then this reason is foolish because if what you want is money then you should go to the City. The money is guaranteed, guaranteed in fact by the government which not only underwrites it, bails it out but then charges off into Europe to protect it from legislation that might affect its lucrative tax haven and money “recycling” business. In contrast being involved in “entrepreneurship” is a rather romantic and significantly more challenging way to achieve wealth.

I do work at a startup though and I was at Silicon Milkroundabout trying to encourage people to join me in doing this.

My personal motivation is that for me a startup is a business that is complete but small enough that you can actually see and understand all parts of it. The interesting thing is that organisational dysfunction is actually just a likely in a startup as a larger firm. Often the problems are actually exactly the same, simply orders of magnitude less significant.

Being able to pull the curtain aside is fascinating. Working in the small also removes the mystique that gathers around things and people that generate large revenues. Once a certain number of livelihood’s become involved in a particular process or product you lose the ability to tinker with things or even to question why things are the way they are. In an environment with no money and no customers any change is either positive or at least neutral.

Working in a startup for non-cynical reasons means creating something that is of profound personal interest. I really am interested in trying to remove friction from the process of turning ideas into reality. Wazoku is a product that I do believe in and what I was saying to a lot of people at Silicon Milkroundabout was try the product. If you are interested in solving the problem and the solution in turn solves some of your problems your work is satisfying at all levels. If there is not a satisfactory solution already in progress for your problem then a startup is the only way that you can initiate that process of moving to a more perfect world.

Working for a startup is a last resort; need should be part of your motivation; ignore idiots and their advice; make sure you get enough sleep.

Finally there was the recruitment activity. Unlike the first event this really was more of a milkround with a younger, less experienced audience. The format did seem to be pitching for talent which is interesting as I am not convinced that people are going to find the best role by going with the best sales pitch. There has to be a better way of understanding the culture of the firm you are potentially joining.

The different streams of activity make the event quite weird in its nature and purposes. It feels like there is a need for a kind of startup expo to allow startups to see and meet one another without the pretext of seeking to employ people. There is also a need for a kind of elite coder event on a quarterly basis that is maybe a little select, a bit like a mini-conference, that allows for networking and swapping of intelligence and gossip on what is really going on at various firms.

Let’s start with an entirely pragmatic answer: the job title is something that is well understood. While the majority of my activities day to day would be adequately covered by the title “lead developer”, the truth is that technical authority and decision making resides solely with myself. The easiest way to convey that to suppliers and recruitment agents (and stop them seeking to go over my head to my non-existent boss) is to use the most commonly understood title.

I find it ironic that I have managed much larger projects with a much junior title. In terms of experience I do not feel a particular gap with other startup CTOs but obviously there is a bigger gap as you move into the equivalent role in larger organisations. A lot of those people come from non-technical backgrounds reflecting the greater need for people management at larger scales. The number of people who have technical backgrounds and have managed groups greater than a hundred people strong onshore are probably pretty small.

For me the key differentiator in my current job is that I do hold a technology portfolio within the management and report on the whole technical area to the management team as well as the board and investors (although to be honest the latter too are not that bothered so far). I would happily concede the “Chief” as I have no other senior technology reportees but I think it is a different kind of pretension that seeks to do down the unique aspects of a role you have in an organisation. I am an officer of the company and I do make the key decisions for technology and I am held to account for them. CTO is the common title and I am comfortable using it.