Product Engineering Blog

Bringing Data Sources Together with PipelineWise

TransferWise is open sourcing it’s data replication framework. PipelineWise is a Data Pipeline Framework using the Singer.io specification to replicate data from various sources to various destinations. Like many companies, the first technology stack at TransferWise was a web page with a

kafka

Achieving high availability with stateful Kafka Streams applications

Kafka Streams is a java library used for analyzing and processing data stored in Apache Kafka. As with any other stream processing framework, it’s capable of doing stateful and/or stateless processing on real-time data. It’s built on top of native Kafka

configuration

Don't write tests against configuration

Why writing tests against production configuration is usually not that good idea and what to do instead.

design

Where to put application configuration?

What is a good approach to configuration? I will compare drawbacks/benefits of externalizing config vs hardcoding config in code.

microservices

What is a good size for a microservice?

To be able to answer that we should start from a different question: what kind of problems we believe smaller/independently deployable codebases solve better than a monolithic codebase with

spring boot

Building modular apps using Spring

When looking at most of the sample applications and code examples for Spring framework it may seem that everything should be defined as a bean. However, if we want to

javascript

Reducing your Webpack bundle

The last couple of months, my team at Transferwise, has been working to deliver a new feature on our product. We call this feature Money Tracker and helps TransferWise users to track their money better. I've been developing it for the web using Angular

Pull Requests that make code reviewers smile

Pull Requests (PR) are central to most software development organisations these days. Modern source control tools such as Github or Continuous Integration ones like CircleCI have helped a lot making them more accessible. Yet, it seems developers consider PRs mostly as something that you

hibernate

Hibernate and Domain Model Design

When I first learned about Hibernate about 12 years ago (I guess just about when it was at its peak) I found it to be one of the coolest libraries

A Product Manager’s guide to planning for the team

One of the best things about working at TransferWise, is how seriously we take the autonomy of our teams. Each team can (and needs to) set its own Key Performance Indicators (KPIs), come up with a mission statement, and a vision for the team.

On the Logging in Mobile Applications

Here I describe an approach from iOS point of view. An approach for Android platform is no different. I can recommend to use Timber framework as an equivalent for CocoaLumberjack. Logs? Phew! Just write something like: print("Expected `foo` to be 'bar', got:

4 fallacious reasons why we estimate

‘Estimate’ as defined in this article is: A guess about the future regarding the time or effort that a task(s) will take to be completed. In other words, anything that can be used to foresee when tasks will be done. I have to

postgresql

pg_ninja, now open source

In my previous post I wrote on how we scaled the analytics database using PostgreSQL. One of the key requirements was the possibility to replicate and obfuscate the data in real time from our main MySQL database. The tool which is managing this particular

When Groovy, assert and threads gang together to fool you

At TransferWise, we use Kafka for our messaging needs, to communicate between our micro-services and with our legacy Grails monolith. A few days ago, I faced the strangest problem with the Kafka message processing in Grails. The code that consumes Kafka messages works in

design

Illusion of Reuse

I have quite often seen situation where trying to achieve more reuse actually ends up in building the Big Ball of Mud. In some cases it is the Enterprise Domain

mysql

Scaling our analytics database

Business intelligence is at the core of any great company, and Transferwise is no exception. When I started my job as a data engineer in July 2016 my initial task was to solve a long running issue with the database used for the analytic

tdd

5 Tips for Getting More Out of Your Unit Tests

State of Application Design In vast majority of applications I have seen the domain logic is implemented using a set of Service classes (Transaction Scripts). Majority of these are based

technology

When to Adopt the Next Cool Technology?

What should be the criteria for an organization to decide when is it a good time to update its toolbox? Recently there has been a lot of discussion about the fatigue around JavaScript and frontend tools in general. Although it seems to be more

Engineering Culture

Building TransferWise or the road to a product engineer

Soon it is my 5 years anniversary at TransferWise. I looked back. I wrote down what has come to my mind. I was hired as an engineer. I thought I was hired to write the code and that is what I started doing. Simple

microapps

Effective Reuse on Frontend

In my previous post I discussed cost of reuse and some strategies how to deal with it on the backend. What about frontend? In terms of reuse both are very similar to each other. When we have more than just a few teams regularly

Product Engineering

Product Engineering Principles @ TransferWise

At TransferWise, we have seen phenomenal growth in the last few years - growth in users, transaction volumes, and team. Our engineering team has grown from 50 to 120 in

microservices

Why Over-Reusing is Bad

One of the Holy Grails of software development has always been reuse. In the following post I want to focus on reuse of application logic in its different forms. I will not cover reuse on more abstract levels like ideas or patterns. This will

grails

The TransferWise Stack - heartbeat of our little revolution

As any tech startup that's passed its five-year mark, TransferWise has come quite a way from the first lines of code that powered it. Our product is a living organism,

exceptions

When to Throw an Exception

Simplest rule to figure out whether we need an exception is to think if having a stracktrace will give any value. For example, do we care about stacktrace when some business rule is violated? One widely known principle is that exceptions should be used

javascript

How to Create Data Model Objects in JavaScript

Why it makes sense to use some special facility (constructor function, factory method, class in ES6) for creating objects of Data Model[1] instead of just using object literals and whatever is returned as JSON from server? Data Model objects typically don't have much