- Reading, United Kingdom
I’m an experienced Senior Software Engineer who has work in both the public and private sectors. I’ve branched into many areas of software engineering and a variety of languages such as NodeJS, PHP, Golang, Scala and Java. I’ve used and can happily work with various development. Be it agile, scrum, kanban or just winging it.
Over the past 5 years I have also scaled NodeJS for DataSift’s real-time stream service that can handle many thousands of concurrent users that were each consuming from the Twitter firehose (up to 12,000 tweets delivered a second each).
Tutsplus, “The best way to learn creative and technical skills”, approached me about creating some Web Development courses. My Introduction to Node.JS course has had 100% positive feedback.
In the past when I worked in the public sector I’ve done GIS (Geographical Information Systems) work for Dorset County Council including the internationally recognised Dorset Explorer http://explorer.geowessex.com
Like: node.js, go, php, mysql, zeromq, redis, git, chef, go-cd, nginx, apache, haproxy
Engineering Team Lead, DataSift, May 2014 – Current
node.js, go, php, scala, redis, apache-kafka, chef, go-cd
After proving myself with the DataSift Push Connectors my skills were recognised and I was promoted to an Engineering Team Lead, which then brought in 3 developers for me to manage and look after. These engineers were spread across multiple features within the DataSift platform.
I still carried on writing software as before, but began to focus in on a new area of the architecture which focused on receiving data from customers, rather than public API’s from social media services like Twitter.
We created Open Data Processing which would allow DataSift to receive a wide variety of data from customers, map it to our internal “interactions”, augment, join, process and filter the data and then re-delivery the enhanced data back to the customer.
Web Developer, DataSift, January 2011 – May 2013
php, node.js, zeromq, apache-kafka, jenkins, mysql, mongodb, sftp, postgresql
Over my time as a Web Developer at DataSift I was focused on producing and maintaining microservices for the service-oriented platform. Highlights of these include processing the Twitter firehose, which reached up to 20,000 tweets a second that were all processed, augmented and filtered in real-time within 0.2 seconds.
The number of customers using the platform was increasing all the time and I was involved with completely re-writing and scaling out the Streaming API. There were thousands of connected streams each potentially receiving several thousand pieces of data every second.
Later on I lead a team of engineers to create new ways to delivery data to customers. These were known as the “Push Connectors”. They connect to customers S3 accounts, SFTP servers, HTTP servers and even MySQL databases to push data to them with very little configuration. This is very complex system that makes use of buffers and scheduling algorithms to ensure the customers is being delivered as much data, as efficiently as possible.
GIS Web Developer, Dorset County Council, October 2007 – December 2010
I worked in the public sector I’ve done GIS (Geographical Information Systems) work for Dorset County Council including the internationally recognised Dorset Explorer http://explorer.geowessex.com
Aside from this work I was developing extensions for MapInfo written in a VB6-type language called MapBasic. This then lead me on to VB.NET and other web technologies
GIS/IT Advanced Apprentice, Dorset County Council, October 2004 – October 2007
mapinfo, asp-classic, asp.net, asp.net-mvc, iis, gis
My first major project was Dorset Explorer, a GIS web application (http://explorer.geowessex.com) that I created on my own and is now adopted by the Environment Agency as part of their national flood incident system.
During this time as an apprentice I completed my Foundation Degree in Internet Technologies
BSc Open Degree, Open University, 2009 – 2012
java, mysql, html, css, apache, nginx
The degree was a combination of modules in:
- Web server management, performance and tuning
- Databases within web design
- Relational databases, theory and practice
- Open Source development tools
- Technologies for digital media
- Distributed Systems
FdSc Computing and Internet Technology, University Centre Yeovil, 2006 – 2009
html, css, php, perl
The foundation degree is part-time and covered various topics such as
- Web application development
- network design
- systems analysis
I graduated with a distinction.