As software engineers, we have always felt it was important for us to have full control over each aspect of our solution, from the API layer, Security, Data layer, Configuration, etc. There are many reasons for this, one of them being some of us started building software pre-AWS which meant to build a full solution we needed end-to-end skills to make even the smallest POC come alive, for others it could just be ego.
However for every curious engineer when AWS was introduced we were itching to get started, but we all fell into the same trap, looking for a quick win which usually meant lifting and shifting one of our client applications to use the IAAS or PAAS tools provided by AWS. This usually meant for some they have learned enough and stopped focusing on digging deeper into the services on offer.
Building a new innovative platform can come at a high technical cost if the lead engineer uses a tech stack that could cripple the company in the future. For many tech-enabled startups, the core of the technology being built usually lies in a few workflows that solve a user’s problems. We have heard many times a lead engineer opts for a specific technology because it is what they know, not because it is the best option. Despite the steep learning curve for the serverless services provided by AWS, they can provide a startup or business a huge speed advantage in getting to the market.
Traditional vs Cloud
The traditional architectural pattern most of us have used has often dictated the underlying technology stack. The popular architectural patterns for web/mobile applications are:
- MVC:
- With certain technology stacks usually taking the lead, c#/.net, java/springboot, php(laravel, cake) and whichever front end technology is chosen
- https://ducmanhphan.github.io/2019-07-27-MVC-architecture-pattern/
- N-Tier/Layered: Presentation Tier/Logic Tier/Data Tier:
- With certain technology stacks usually taking the lead, c#/.net, java/springboot, php(laravel, cake) and whichever front end technology is chosen
- https://stackify.com/n-tier-architecture/
Now the technologies are very solid and reliable and have allowed engineers to build world-renowned software, they have a steep learning curve, however, the challenge we see in them is more around how much software should be written by a developer to build a POC that is still validating whether the technology/business itself has a market worth going after.
This means our focus shifts from, what are the tools I know to what are the tools that will help me get a solid POC together. This means as an engineer we need to find out where the magic or real business values sits within the application. For most tech-enabled businesses this usually means the business logic, integrations, and user experience usually become the core of what we are trying to test as a startup. That means the code we used to write for our API layer, Security, Filters, etc can be outsourced to a serverless service that has little or minimal integration requirements. This means our focus should be on how we deliver the business logic, integrations, and user experience required for our end users.
Use Case: Video endorsement platform using AWS services
The best way we learn is always through building and improving over time. So the use case we will introduce below is to demonstrate how we built a video processing platform, with a mobile app as the front end and AWS powering the entire backend.
The above architectural diagram utilizes many AWS services, the core process/user experience and integrations highlighted above is focused on a specific part of the user journey which is to allow the user from a mobile app to upload videos to be processed (cropped, watermarked, resized for distribution and generating thumbnails) and linked back to the users account for sharing. We have deliberately left out many of the other business logic to demonstrate the value of using cloud services to handle the non-core components.
Below is a brief description of the service and its usage:
- API Layer:
- AWS API Gateway, the benefit of using this instead of writing the code was we were able to build in security, our models, exception handling, and restful responses. This also acted as a proxy that invoked the actual business logic lambda that done the actual work.
- AWS Cognito, the benefit was getting user sign-up, sign-in, and access control to our application without the need to write the additional code
- Logic Layer:
- AWS Lambda, this a serverless solution that was invoked either by an API Gateway endpoint or a trigger from an event within our video pipeline.
- AWS SES, is a service allowing us to send notification emails out to our end users for certain events within the application
- Data Layer:
- S3 was used to store video files throughout the video pipeline journey and for static images linked to the user’s profile
- DocumentDB was used as our core database store
Pros of the above approach:
- We were able to build a mobile app, the necessary business logic, and video processing pipeline in 2 months
- The cost of processing over 100 videos has cost us roughly $80 a month
- Adding a second developer to the project was fairly easy as the approach to building out new endpoints were simple
Cons of the above approach:
- A lot of moving parts, which could become difficult to manage over time without proper documentation and knowledge sharing
- DevOps was ignored in the initial stages, which meant manual deployments for each function(50+)
- Connecting directly to a database from a lambda function has its pitfalls due to the cold start issue, which leads to performance issues if the application is idle
Overall the experience above has led us to believe the approach to building POCs needs to change, as the value founders/businesses are looking for is being able to get something to market as quickly as possible to be able to validate whether the technology can provide value or not. Going from POC to the scalable solution is also not too difficult as the majority of what’s needed to shift to a scalable solution is handled by the various services.
Try out Skywalk
Would you like to explore building a POC on AWS? Give Skywalk a chance. Book a no-obligation session with our team and explore more ideas around technology to grow your business.