What is Platform Engineering
Platform engineering is an approach to building and managing technology platforms within an organization. A platform provides a common foundation that software products and services that can be built on top of. Platform engineers focus on creating tools, services, and infrastructure that serve as the core technology other engineers use to develop applications and software.
Key Responsibilities of Platform Engineers
Some key responsibilities of platform engineers on a platform team include:
- Developing shared services and APIs that can be reused across multiple products/teams
- Creating developer tools and systems to improve engineering productivity
- Building infrastructure and automation to simplify deployments and operations
- Establishing patterns, standards, and best practices for how software gets built and run
- Providing self-service access to technology resources for engineers
- Managing and monitoring platform systems for reliability, scalability, and security
Platform engineering focuses on internal customers (engineers within the same company) as opposed to external customers. The goal is to abstract away common complexity and provide configurable building blocks that engineers can use to deliver products and features faster. Well-designed platforms allow engineers to focus on differentiating business logic rather than reinventing foundational technology.
Why Platform Engineering Matters
Platform engineering and having an internal developer platform strategy are becoming increasingly important for software engineering organizations building software products and applications. Here are some of the key benefits of investing in platform engineering capabilities:
Accelerate Product Delivery
Platform teams can significantly accelerate feature development by providing reusable components and services for development teams. Engineering teams don't have to rebuild foundational technology for every product and can focus on their specific problem domain. A platform removes friction and burden from product teams and lets them concentrate on innovation and customer value.
Drive Engineering Efficiency
Platforms enable engineers to be more productive by abstracting away infrastructure complexity and providing self-service access to resources. Teams spend less time on mundane tasks like managing servers or configuring monitoring and can dedicate more cycles to advancing the product.
Promote Best Practices
An effective platform promotes standards that drive quality and consistency across engineering projects. This includes establishing common design patterns, defining project architecture principles, standardizing coding practices, and enforcing security policies. The result is more reliable and robust applications.
A platform provides a catalog of shared services that multiple teams can discover and leverage. This increased interdependency fosters better communication and collaboration across products and groups. The platform creates synergies across the engineering organization.
Recruit and Retain Talent
Engineers want to focus on solving interesting problems and building products, not infrastructure. A company that invests in its platform and engineer experience attracts top talent and enables the organization to produce better work.
Achieve Economies of Scale
There are major cost efficiencies in reusing platform infrastructure across engineering teams versus fragmented systems built redundantly. Companies can achieve economies of scale from both a cost and productivity perspective.
Elements of a Strong Platform
Robust developer platforms involve multiple components working together to maximize productivity. Here are some key elements:
High-quality developer tools boost efficiency. This includes IDEs, CI/CD, observability, source control, and workflow systems like JIRA. Self-service access makes onboarding frictionless.
Reusable software services provide common functionality via APIs that teams can leverage:
- Data and storage - Databases, object storage, caching, search, stream processing
- Messaging - Async task queues, message buses, notifications
- Security - Authentication, authorization, secrets management
- Serverless - Functions like AWS Lambda to run isolated code
- Machine learning - Shared models, prediction services
- Media processing - Transcoding, thumbnailing, watermarking
Foundational Building Blocks
Core platform foundations that provide baseline functionality:
- Data - Data stores, pipelines, lineage, governance
- Infrastructure - Networking, runtimes, containers, resource management
- Observability - Logging, tracing, metrics, alerting
- Security - Encryption, certificates, secrets, RBAC
- Testing - Test automation frameworks, mocks, performance
Internal Developer Experience
Optimizing the experience for internal engineers:
- Self-service - On-demand provisioning of resources
- Developer portal - Documentation, guides, API reference
- Community - Communication channels, mailing lists, chat
- Onboarding - Developer training, support, office hours
- Request management - User feedback, feature requests, roadmap input
- Status transparency - System dashboards, incident reports
Challenges in Platform Engineering
Building a robust internal developer platform presents challenges including:
Adoption - Getting teams to leverage shared services vs going rogue. Must demonstrate value.
Funding - Leadership must see the long-term benefits of platform investment.
Fragmentation - Siloed teams duplicating effort without coordination.
Communication - Facilitating cross-team discussions and gathering feedback.
Talent - Platform skills span software engineering, ops, and product. Difficult combination.
Security and compliance - Added risks and responsibilities inherited from the services built atop the platform.
Ownership - Should the platform be owned centrally or by individual teams?
Complexity - Managing complexity as platforms scale. Avoiding technical debt.
Evolution - Incremental changes while maintaining legacy support. Avoiding big bang rewrites.
Focus - Balancing custom platform services vs leveraging managed cloud services.
Best Practices for Platform Success
Here are some best practices that can lead to effective platform implementations:
- Start small - Deliver minimum viable capabilities, expand incrementally
- Developer experience - Should make engineers' lives easier, not harder
- Collaboration - Close communication between platform and product teams
- Customer obsession - Solve internal engineers' pain points, don't chase fads
- Leverage cloud services - Don't reinvent the wheel unnecessarily
- Automate relentlessly - Manual processes don't scale
- Standardize judiciously - Some standards enable agility long term
- Central control vs autonomy - Balance governance with developer freedom
- Observability first - Logging, metrics, and tracing pave the path
- Loosely coupled architecture - Isolate services for independent evolution
- Plan for growth - Build with expected future scale in mind
There are a few common organizational structures for platform teams:
Centralized Platform Team
A dedicated cross-functional team owns the core developer platform from top to bottom. They provide services to other engineering teams but don't build end-user features.
Embedded Platform Engineers
Platform engineers integrate directly into product feature teams rather than a separate central team. Still, coordinate standards across the organization.
A central platform team governs the overall strategy and foundational services while embedded engineers fill gaps on specific teams as needed.
Developer Productivity Teams
Some engineering organizations have teams entirely focused on improving developer productivity, tooling, and infrastructure.
Roles on Platform Teams
Here are typical roles that contribute to platform teams:
- Software Engineers - Build platform components across the stack from frontend to infrastructure.
- Product Managers - Gather requirements, define roadmaps, and set priorities for platform capabilities.
- Developer Advocates - Promote platform adoption, support onboarding, and foster community.
- Solutions Architects - Establish high-level architecture strategy and guide technology choices.
- SRE/DevOps Engineers - Deploy, operationalize, and monitor services. Ensure reliability and uptime.
- Data Engineers - Build pipelines and infrastructure for analytics, reporting, and machine learning.
- Security Engineers - Provide security guidance, enforce policies, and ensure platform integrity.
- QA Engineers - Automated testing, release validation, defect management.
While titles vary, platform teams need a mix of software engineering, operations, and product management skill sets.
Important metrics to track how well a platform team is achieving its goals:
- Adoption - % of teams using platform services, growth in usage over time
- Satisfaction - Developer experience ratings collected via surveys, interviews
- Productivity - Improved team output or velocity since the platform rollout
- Reliability - Change failure rate, service uptime, performance
- Time to market - Reduction in project kickoff to launch due to platform leverage
- Cost efficiency - Decrease in redundant spending on infrastructure, tooling
The Platform Engineering Process
Here are some best practices for approaching platform builds:
Understand Internal Customers
Get insight into engineers' pain points through techniques like value stream mapping and jobs-to-be-done. Empathize with their realities.
Start with High Leverage Problems
Identify the areas that will have the biggest impact on developer velocity and focus on solving those first.
Design with Developer Experience in Mind
APIs, SDKs, and tooling should be intuitive and frictionless for engineers. If not, they won't adopt them.
Maintain Close Communication
Keep a feedback loop between platform teams and the consumers of platform services to continually align.
Plan for Evolution
Build in flexibility for changing requirements and support for legacy consumers over time.
Invest in Automation and Observability
Manual processes don't scale. Logging, metrics, and tracing pave the path to reliability.
Embrace Compliance as Code
Make compliance checks part of CI/CD pipelines to bake security in from the start.
Build Reusable Abstractions
Design components that abstract complexity away from consumers while remaining customizable under the hood.
Stay up to date on new technologies, standards, and practices to inform platform roadmap priorities.
The Emergence of Platform Engineering Teams
As software engineering organizations grow, the need emerges for an internal team dedicated to maximizing developer productivity. Reasons platform teams arise:
- Engineering teams start spending too much time on undifferentiated infrastructure versus core product work.
- Duplicated effort across teams from not sharing common solutions.
- Difficulty onboarding and ramping up new engineers without standardized tooling and stacks.
- Unreliable, slow legacy systems hinder developer velocity. Lack of automation, testing, and monitoring.
- Security and compliance risks increase with fragmented systems and inconsistent practices.
- Developer experience suffers when technical roadblocks impede product innovation.
The platform team addresses these issues by providing reusable building blocks that reduce friction. They aim to remove distractions so engineers can focus on delivering differentiated value. Mature engineering organizations view platform investment as a force multiplier that pays dividends over time.
The Future of Platform Engineering
As technology continues evolving rapidly, platform engineering will only increase in importance for software organizations. Here are some key trends shaping the future landscape:
- Everything as Code - Platform tooling will enable full environments to be defined and managed as code.
- Increased Adoption of Cloud Services - More reuse of managed cloud infrastructure vs custom platform builds.
- Rise of Internal Developer Portals - Self-service dev portals for documentation, sandboxes, stacks, and tooling.
- Focus on Security and Compliance - Greater need for security automation, policy enforcement, and audit trails.
- Value Stream Optimization - Platforms will facilitate easier tracking and optimization of engineering value streams.
- Mainstreaming of Site Reliability Principles - SRE skills expected from all platform engineers.
- Low-Code Development - Platforms will empower low-code solutions and citizen development where optimal.
- Everything Gets Instrumented - Increased telemetry across all systems, apps, and processes.
To attract top talent and accelerate innovation, investing in internal developer platforms will be key for ambitious engineering organizations. Platform capabilities provide the foundation for engineering teams to thrive and maximize their productivity over the long term.
- Platform engineering focuses on enabling engineering productivity through reusable services and infrastructure.
- Platform teams aim to remove the friction that slows down product teams so they can focus on innovation.
- Effective platforms provide self-service access to data, security, CI/CD, ML models, and other utilities via APIs.
- Platform capabilities should optimize for internal developer experience.
- Organizations have flexibility in how they structure platform teams based on their needs and culture.
- Platform engineering requires a combination of technical and product management skills to understand internal customer needs.
- Investing in platform engineering and internal developer platforms provides a competitive advantage by accelerating developer velocity across the engineering organization.