Bitpanda's new trade engine: Part #2 - Optimising language choices & frameworks

How to pick the right programming language

These days there are dozens of languages to choose from for your new applications within your ecosystem. While many of these languages will work well even with a modest load, only a few of them will perform exceptionally when chosen for their intended purpose.

Though it is nearly impossible for an engineer to have expertise in every available programming language out there, it’s important to not restrict yourself to only one. Diversifying your knowledge allows you to explore and tap into the potential of what other programming languages have to offer. 

That’s why in the ever-evolving landscape of technology, we understand the importance of being polyglot programmers, albeit within defined boundaries. We defined a clear strategy of the languages we endorse by using Techradar, establishing concrete guidelines on what is and is not allowed to be used for a new microservice. This tool evolves over time and, as the main goals of a company change, so do the languages and frameworks.

Our language selection process

Our team, the trade engine, currently consists of 8 backend developers. While there would be nothing wrong with using one language to write all of our services, the increased team size allows us to experiment with multiple languages for different purposes.

Before revealing what languages we use to write our applications, let’s analyse the benefits of being a polygot team:

• Team member growth and learning: Learning a new programming language exposes us to different paradigms and design patterns. This allows us to be better prepared for solving complex problems that require us to choose the best technology for the job. Ultimately, we are valued for our expertise, so it is essential to continuously update and expand our knowledge.

• Faster system: Choosing a programming language that supports multithreading is an excellent fit for tasks that can be performed in parallel.

• Quicker delivery: Working with highly mature frameworks that require only a few lines of code to add CRUD endpoints for your domain entity can significantly reduce the time-to-market.

• Ecosystem and tooling: Not every technology is available for every programming language. At times, certain languages are better suited for interacting with specific APIs or databases. For example, if you intend to use Kafka Streams, you would typically choose Java. Similarly, when working with a robust Cassandra client SDK, PHP might not offer suitable options, as there might be no PHP client SDKs that currently support it.

On the other hand, our team size can also limit us and there could be some disadvantages, including: 

• Fragmented expertise: Every team member has experience and skillset in various programming languages. In situations where a high-priority issue arises, it's essential to recognise that the delivery times may not be equal across all team members. Some engineers may require more time than others to address the issue effectively. To address this challenge, we've shifted our hiring focus. Instead of restricting ourselves to developers bound to specific languages, we look for versatile individuals. Our aim is to find professionals capable of solving problems using different technologies.

• On-call difficulties: What if your team is not in a language-polyglot state? Let’s say your Java engineers are on vacation and you facing severe issues in your production, who is going to fix them? In most cases at Bitpanda, bugs were resolved by reverting to a previous version. Sometimes even that is not required and just simply turning a feature flag off will solve the issue. Go ahead and read a fantastic article by our colleague about improving software releases by using feature flags.

• Maintenance: We have all seen it. We upgrade the SDK version and, until we get it to production, there is a medium-to-high chance that there is already a newer version available. The pace of development of open-source packages is high, so we need to keep up-to-date at a similar pace. The more languages we have, the more version upgrades we need to manage and therefore the running costs are higher.

• Security patches: At Bitpanda, we have a dedicated DevSecOps department to handle this task. They ensure the safety of our base Docker images for each programming language. As we support more languages, the workload of keeping these images up-to-date increases.

• Hiring difficulties: Hiring a Go developer typically takes more time compared to hiring a Java developer. Additionally, as we support more languages, preparing different interview tests for each language becomes more time-consuming. In some instances, it may also require additional financial resources if we opt for third-party providers to conduct these tests which also contribute to increased costs.

Why we chose PHP, Java, and node.js

There are no clear guidelines that dictate the exact number of programming languages a team should use in production. The decision depends on various factors, including the team's budget, maturity, seniority, and willingness to learn new skills and technologies. It is important not to overwhelm potential new team members with an excessively diverse stack. At Bitpanda, we understand the significance of the work we do and acknowledge that having a continuous desire to learn and explore new technologies is an inherent part of our DNA.

In the beginning, our team consisted solely of PHP developers, so our initial microservices were naturally built using PHP. As our needs grew, especially for robust public APIs supporting GraphQL, an external company guiding our shift to asynchronous event-driven programming made suggestions and assisted us in building these services using Node.js. Over time, the seek for enhanced performance led us to explore enterprise-grade solutions. We gradually moved away from PHP and embraced Java. To facilitate this transition, we brought in a highly skilled senior engineer adept in Java development. With this diverse team of experienced developers, we now proficiently handle PHP, Node.js, and Java, expanding our capabilities significantly.

Of course, we understand that comparing programming languages is a very complex and often intuitive task. That’s why in this next section we offer a subjective perspective.

PHP

One of the key factors why PHP is in our stack was its integral role in the early versions of Bitpanda systems. A considerable portion of our internal tools and client SDKs were purposefully developed using PHP. Moreover, the majority of software engineers in Bitpanda were hired as PHP development specialists. However, these are not the only factors: 

• Large community:  PHP is widely adopted by many small companies globally, so you'll be able to find answers to most of your current web development issues. With an active developer community, PHP contributes extensively to various open-source projects, providing a wealth of resources for assistance.

• Easy to learn: The syntax is straightforward and resembles other programming languages, making it relatively easy for newcomers to grasp and start building web applications quickly.

• Rapid development using mature frameworks: Opting for a highly mature framework that streamlines much of the language complexity allows you to begin managing new requests within just an hour of bootstrapping your project.

• Continuous development: PHP gets a major release roughly every 2 to 3 years with new hefty features contributed by the community.

Java

As your company grows into a large entity building complex distributed systems, PHP might not keep pace with more mature languages. This is why we introduced Java to our stack as it is perfectly suited for large-scale enterprises, providing us with these extra benefits:

• Strong type checking and static typing: This in itself significantly decreases the chances of runtime errors and unforeseen application behaviours, as many issues can be detected during the compilation process. Java is known for its robust performance owing to its compiled nature and sophisticated optimisation capabilities caught during compilation time.  

• Faster on computationally intensive tasks: Java offers strong support for multithreading and concurrency, allowing our applications to perform computationally intensive tasks simultaneously, including crucial exposure validations.

• Best support for Kafka: Apache Kafka oversees the development of the official client library, granting us access to the full spectrum of features that Kafka offers, including Kafka Streams.

• Mature frameworks: Frameworks like Spring Boot and Micronaut have been in the market for many years. They have undergone rigorous testing in real-world scenarios and have been embraced by numerous enterprises, making them an excellent fit for our architecture.

Node.js

Our stack also includes support for Node.js, the third programming language. As stated in one of the previous sections, we needed a good candidate for our robust public APIs supporting GraphQL. Node.js was suggested by an external company we closely worked with that guided our shift to asynchronous event-driven programming. After using it for some time, we can say Node.js strength lies in:

• Event loop: Node.js single-threaded, non-blocking, event-driven architecture enables us to function seamlessly even with hundreds of concurrent connections. This quality makes it an ideal match for our Public API gateways or proxies to our services.

• Extremely wide adoption: It has a rich and rapidly growing ecosystem of open-source packages and modules available. These packages cover a wide range of functionalities, allowing us to very easily work with external APIs.

As previously mentioned, all three languages can do the job, but it’s important to understand their details and intricacies. Knowing when to opt for each is crucial. In my opinion, both Java and Node.js are capable of achieving what PHP does, but the distinction lies between raw Java and Node.js. Node.js excels in handling I/O-intensive tasks, whereas Java shines in managing CPU-intensive tasks.

After reading this section about the programming languages we selected, a careful reader would notice that PHP and Node.js are not on the list of languages we endorse via Techradar. That would be an astute observation. As time progressed and Bitpanda shifted to being an enterprise, PHP was no longer a good candidate. Also, with more Java experience and maturity in the team, we believe a public API written in Java would have done just as good a job as one written in Node.js. Due to recent macroeconomic factors, it has become more difficult to support a varied palette of programming languages and the costs of supporting multiple programming languages outweigh the benefits, therefore we see this as a slip.

Choosing the right framework

After you decide to extend your list of supported languages with a new programming language, my suggestion is to always bootstrap your new project using a mature framework and I believe it is fairly straightforward to pick one.

Laravel

Across our company, Laravel stands as the most favoured choice. It offers an elegant syntax which abstracts away a lot of unnecessary boilerplate code, streamlining the coding processes and boosting our efficiency. If you need to have absolute control over your code and optimise every bit of the application, then you will need to delve into the “magic” that Laravel comes with, or give up and try another framework such as Symfony.

However, if you are comfortable with this arrangement, Laravel may surprise you with many out-of-the-box integrations for many different databases (MySQL, PostgreSQL, Redis), querying systems (AWS SQS), file systems (AWS S3) and much more. If you are developing an API, attaching a custom middleware to your set of routes is a piece of cake. And did we mention that you can apply a rate limit on your endpoints with just one line?

Micronaut

Within our solution, we opted for Micronaut – a somewhat less mainstream choice compared to Spring Boot. When making this decision, we were drawn to using a rather more lightweight Java framework that has quicker startup time, reduced memory footprint and better resource usage as in our cloud environment it can have a significant impact on a final monthly bill.

What is appealing about Micronaut is that it was built from the ground up with cloud-native thought in mind by offloading some optimisations to the build time by removing reflections that cause runtime overhead. Moreover, the framework includes Netty – a reactive application framework – granting us non-blocking I/O capabilities.

In the end, Micronaut offers everything necessary to develop an enterprise-level application. Should the need arise, integration with Spring modules is a simple process, ensuring you're never left stranded.

Express.js

When you start researching the best Node.js framework, you will be blasted with tons of suggestions for the Express.js framework and how good it is. This alone gave us the impression of a mature, widely embraced, and rigorously tested framework.

At present, we can confidently say that it was the right decision, primarily attributed to its simplicity, minimalistic nature, and flexibility it offers in incorporating external libraries such as GraphQL or Kafka-js.

Summary 

Overall, there are no clear answers to the question of the best programming language, and in my opinion, there will never be one. There will always be a personal preference towards one language over another and this can play a significant role. Therefore, based on my current experience and observations, PHP can be a good fit for many small to medium-sized companies offering their products to tens of thousands of users. 

Once your company grows to medium to large size and offers its products to hundreds of thousands of users, you will likely be better off choosing a more robust language such as Java, which has been proving its high position among top enterprise languages for decades.