Technical Articles

AWS Cloud practitioner basics

1 Point ContributorLeave a comment

Introduction:

  • Cloud practitioner overview is for non-technical roles. It helps taking business decisions of choosing a cloud solution. Cloud is a pay as you need solution.
  • On premises deployment is also known as a private cloud deployment.
  • Hybrid deployment: Hybrid environment has both, on premises and cloud deployments. Maybe for some applications, it makes sense to use on premises solution and for some, it makes sense to use a cloud deployment.
  • Amazon Athena: It is a query service to query S3 data, using standard SQL.
  • Cloud search: It is a search service that can be implemented for an application.
  • Container orchestration tools: ECS and EKS. Both use dockers for containers.
  • AWS Fargate: It is a serverless compute platform for ECS or EKS. In ECS and EKS, we need to manage servers. In Fargate, servers are managed by Amazon.

Global Infrastructure and Reliability:

  • Regions: AWS has multiple regions. No data goes to another region by design. There are ways to allow data access across regions. Four factors to choose a right region: compliance, proximity, feature availability, and pricing.
  • Availability zones: Regions contains Availability Zones. Regions are not at one location. Each region is made of multiple data centers. Ann availability zone has a single or group of data centers.
  • To maintain fault tolerance, it’s recommended to run applications with at least two availability zones in a region.
  • Edge locations: Edge locations run Amazon Cloudfronts. Amazon Cloudfronts these are global Content Delivery Networks (CDNs) of Amazon. It uses edge locations to accelerate content deliveries. We can use these to cache the data closed to customers locations. Edge locations also run DNS services called Amazon route 53.
  • AWS Outposts: These are isolated infrastructures within a company’s building.

Ways to interact with AWS:

  • AWS management console, AWS Command Line Interface (CLI), AWS Software Development Kits (SDKs), and other tools (like Cloud formation).
  • AWS management console are good for beginners.
  • AWS Command Line Interface (CLI) are better for production environment. using CLI, we can invoke commands to interact with terminal instances and automate processes.
  • AWS Software Development Kits (SDKs) allow to interact using various programming languages.
  • Other tools:
    • AWS Beanstalk allows us to let AWS create the desired infrastructure.
    • CloudFormation: allows us to manage infrastructure as code, to define configurations of the infrastructure.

Networking:

  • Amazon Virtual Private Cloud (VPC). Public subnet: an area that is open to public. Private subnet is not open to public. Internet gateway is kind of a door that allows selected network to be inside the network. VPC allows the private traffic to get inside the network. It doesn’t allow public network to get inside the network. AWS Direct Connect provides a direct connection to your network.
  • Public subnets contain resources that should be accessible by the public. Private subnets contain resources that must be accessible only through the private network, like a database.
  • Access Control List (ACLs) protect the subnet.
  • Security groups protect the individual server instances as security groups are at a server instance level.
  • Security group is stateful. It remembers who is allowed to enter into a server instance. ACLs are stateless.
  • DNS Route service: it helps to provide an IP address of a website.

Storage and databases:

  • Instance stores and Amazon Elastic Block Store (EBS): Block stores are blocks to store files. Hard drive of the computer is an example of a block store. With EBS, we can create virtual hard drives. Incremental backups on EBS are called snapshots.
  • Amazon Simple Storage Service (S3): At S3, we can store files. We store files in buckets. 5TB is the maximum size of a file/object. S3 standard provides very high availability. It stores data at least three facilities. S3 Standard infrequent (S3 IA) is suitable to store files that needs less frequently. S3 glacier can be used to store archives that are rarely to be accessed. AWS supports more S3 types for the various types of data storage needs.
  • EBS versus S3:
    • S3 is serverless. We don’t need any EC2 or another instance, to manage S3 buckets/storage.
    • If we need multiple updates, EBS is a better choice. If we’re changing complete objects, S3 is a better choice.
  • Amazon Elastic File Systems (EFS): With EFS, we can keep existing file systems.
  • EBS versus EFS: EBS volumes are attached to EBS instance. EFS can have multiple instances used at the same time. EBS stores data in a single availability zone. EFS is a regional resource.
  • Amazon Relational Database Service (RDS): It supports a variety of databases.
  • Amazon Aurora: it is compatible with MySQL and PostgreSQL at a 1/10th of the cost.
  • Amazon DynamoDB: It is a serverless and a non-relational NoSQL key-value database.
  • RDS versus DynamoDB: For complex relational joins for analytics, use RDS. For single table types data, key-value is a good choice.
  • Amazon Redshift: It is a data warehousing service that can be used for big data analytics.
  • AWS Database Migration Service (AWS DMS): This service helps to migrate databases on a one time or a continuous migrations models. For example, this service can help to migrate a MySQL database to an Amazon Aurora database. This service can also help to consolidate multiple databases to a single database.
  • Additional Database Services: There are other database services supported by AWS. For example:
    • Amazon Neptune is a graph database.
    • Amazon Document DB for document databases that supports MongoDB workloads.
    • Amazon Quantum Ledger database (QLDB) is a ledger database. For example, QLDB can help to review a history of all changes made to an application data.
    • Amazon Managed Blockchain to create and manage blockchain networks.
    • Amazon ElastiCache to add caching layers on top of a database, to improve the read activities.
    • Amazon DynamoDB Accelerator (DAX) is an in-memory cache for DynamoDB.

Security:

  • Shared responsibility model: Both, AWS and AWS developers are responsible for the security.
  • User permission and access:
    • Root user: A root user is an owner that can not be restricted anywhere where they are owners.
    • IAM User: A user can not login by default. A user needs explicit access to any resources. We need to attach an IAM policy to a user, to provide an access to a resource.
    • Role: roles allow or deny the access to a resource.
    • IAM policy: An IAM policy is a document that allows or denies permissions to AWS resources.
    • IAM groups: An IAM group is a collection of IAM users.
  • Compliance: AWS provides ways to be compliant as per the needs within the country or the region of the business.
  • Distributed Denial of Service (DDoS): AWS security features help avoid DDoS attacks.

Monitoring & Analytics:

  • CloudWatch: CloudWatch can helps in monitoring in many ways. For example, we can generate alerts based on some logs.
  • CloudTrail: It’s an API auditing tool. It helps to log user actions entries, to analyze who did what and when.
  • Trusted Advisor: It’s a service that will utilize your applications/resource about cost optimization, performance, security, fault tolerance, and service limits.

Pricing and Support:

  • Free Tier: 12 months free.
  • Billing Dashboard: This is useful to know about the cost of running your AWS infrastructure.
  • Consolidated billing: it allows to view all costs of multiple accounts at one place.
  • AWS Budgets: we can define a budget amount so that we get an alert when budget is reached.
  • Cost Explorer: it shows where are we spending more money so we can track it over time.
  • Support Plans: Basic Support is free. Business support plan includes all AWS trusted advisor checks at a low cost.
  • Five pillars of the well-architected framework: operational excellent, security, reliability, performance efficient, and cost optimization.
  • Marketplace: Here, we can find click and go services.

Migration and Innovation:

  • Cloud Adoption Framework (CAF): CAF can help to migrate resources to AWS.
  • Migration strategies: Six Rs for migration are six possible options for the migration. Six Rs are:
    • Rehosting: Lift and shift.
    • Replatforming: Lift, minor changes, and shift. No code changes.
    • Retire: Do not migrate.
    • Retain: These are applications that maybe about to go end of life. Do not migrate these and plan to decommission these.
    • Repurchasing: Applicable to business who are looking to repurchase new vendor softwares.
    • Refactoring: Write new code. Driven by strong business needs, to write new code. This has highest initial cost.
  • Snow Family: Three sizes below in the order of lower to a higher capacity:
    • Snowcone: contains edge computing. This is used to ship TBs of data to AWS to backup at AWS.
    • Snowball Edge: They fit into existing servers.
    • Snowmobile: Ideal for larger migrations. Up to 100 Petabyte data.
  • Innovations with AWS:
    • VMWare cloud on AWS: We can migrate VMWare based infrastructure on prime can be moved to AWS.
    • SageMaker: It builds and trains ML models using popular open sourced frameworks.
    • Augmented AI: It is for AI solutions.
    • Lex: It is the technology for Alexia.
    • Textract: To extract text and data.
    • DeepRacer: Experiment reinforced learning.
    • Internet Of Things
    • Ground Station

Cloud Journey:

  • Three tier architecture: Load Balancer, EC2, and Databases. Pillars of a well-architected framework:
    • Operational excellence
    • Security
    • Reliability
    • Performance Effienciency
    • Cost Optimization
  • We can use a well architected tool to suggest where are gaps in an application at AWS.
  • Benefits of AWS Cloud: Six main benefits of using AWS cloud:
    • Trade fix expense for variable expense: assess cost for: Physical space, Hardware, Staff for racking & stacking, Overhead of keeping data center running, and Fixed cost.
    • Benefit from massive economies of scale
    • Stop guessing capacity
    • Increase speed and agility
    • Stop spending money running and maintaining data centers
    • Go global in minutes

Reference:

  • AWS Cloud Practitioner Essentials course.
  • AWS documentations.

Other programming terms

1 Point ContributorLeave a comment

Program, Process, and Thread:

  • A program is an executable file. It is stored on the hard disk. For example, MS word software is an executable file.
  • A process is an execution of a software that uses RAM. For example, creating a page on an MS word document is a process.
  • A thread is a part of a process. A thread is an execution of a task within a process. A process can have multiple threads for multiple tasks. For example, On an opened page on an MS word, a thread could be to write the page. On the same page, another thread could be to check the spellings. These two threads could run in parallel within the same process of adding words in a page.

Zookeeper basics

1 Point ContributorLeave a comment
  • It is an open-source coordination service for distributed applications. Zookeeper is used to manage configurations in distributed systems.
  • Leader node is responsible to manage the load across follower nodes.
  • It is a service similar to DNS for nodes.
  • Zookeeper is not a replacement for a Load Balancer.

References:

GraphQL

1 Point ContributorLeave a comment

Introduction: GraphQL is a query language for APIs. In addition to REST APIs, this is another way to query the data via APIs. Refer to the reference material for more details about GraphQL. It’s not an alternative to REST. It’s an additional option, in addition to RPC.

References:

Event sourcing

1 Point ContributorLeave a comment

Event sourcing is a pattern for storing data as events in an append-only log.

Here are important terms about event sourcing:

Command: A command is an action by a user or a system. Commands are usually stored in FIFO queue.

Event: An event can be defined as a change in state. The result of a command is an event. Event is represented as a past tense. Order of event must follow order of command. So, we store events in FIFO queue. Example of events are: order placed, books purchased, etc.

State: State is a change after an event is completed.

State machine: State machine drives the event process. It validates commands and apply events to update the state.

CQRS (Command Query Responsibility Segregation): It is the segregation of commands and queries for a system. As per CQRS, there is one (single) state machine for the write operations. And, there could be multiple state machines for read-only operations.

References:

Server architecture styles

1 Point ContributorLeave a comment

Below are popular architecture styles:

  • Monolithic architectures: It’s an old way of creating physical servers within single blocks.
  • Service oriented architectures: Similar kind of services are deployed at one location.
  • Micro-service architectures: Services in this architecture are more loosely coupled.

References:

How to make current job more fulfilling

1 Point ContributorLeave a comment

Are you losing interest in your current job? Do you feel perhaps changing it will bring more happiness? Before you go further with stronger steps, what if you re-evaluate your situation? Is it really the job you less like or something else within the job that needs adjustments? Let’s review some steps on how we can make our current job more meaningful:

  • Know why behind the job you’re in: Understand why you’re in this job. Is it your financial responsibility that’s bringing you to the work every day? Why did you choose this line of work? Does it relate to your interests?
  • Is there anything that is lacking in the current job? Evaluate reasons what can increase the engagement at work. What growth perspective you’d like to focus on? Is it more salary? Is it the increase in the impact to the people or business you support? Is it the technology/skills you use at work? Here’s a quick trick to evaluate. Would you prefer to continue doing the work you do, if:
    • You’re offered 2 times the current salary or
    • You’re offered a promotion or
    • You’re offered to increase your business impact or
    • You’re offered to work with more people or
    • You’re offered to switch to use different technology/skills
  • Evaluating it would help to increase the engagement. If we find out that we should move for a better option, we should plan for it. Sometimes, we find out that we’re already in a great environment. For example, a friend of mine was not engaged at work. After the analysis, he realized that given two times the salary, he’d continue working at the same place with the same set of people, supporting the same business. So, there was nothing wrong in his current job except a low pay situation. He decided to explore if he’s really paid low. The reality was that he’s reasonably paid. His comparison of low-pay was also somewhat unreasonable. Sometimes, it’s our comparison, judgement, and other such inner-enemies that cause us distress. Sometimes some people have a feeling that we’re not doing enough or we’re simply not happy where we are. After the realization, he’s now focusing on his current job with a desire to increase his impact. Another person in a similar situation found out that he can get a better salary outside. He planned to move to a better paying job.
  • What’s the minimum required from you at work: Make a list of minimum required expectations from you at work. How knowing the minimum required expectations help? Many times, we’re overwhelmed and lost with many demands at work. Sometimes, it becomes confusing to navigate the day at work successfully. Making a list of minimum required tasks provides us the clarity of expectations. For example, if someone is a software developer at work, they can write minimum required tasks as:
    • Prime tasks like driving projects A, B, and, C. Write down what’s the minimum required expectation for each project.
    • Supporting tasks: update on progress every week, month, and quarter. What does it means? How much times does it require every week?
    • Trainings: are there any mandatory trainings that you have to attend?
  • After making a list of minimum required tasks, plan your time at work to first address these tasks. In my experience, I noticed that many times, ambiguous situations and confusions take up a lot of time. Without the clarity of minimum required tasks, we maybe wasting time. After a certain time, we start being overwhelmed with many things undone.
  • Look at the bigger picture of your current job. Let’s look at three areas to evaluate it:
    • People: how are your work relationships with people? Do you feel connected with people at work? Can you make friends at work to share your life and listen to their life situations? Connecting with people at work can make the work more interesting. Also, knowing that you’re not alone in life’s common challenges can help to find mentor support. For example, a friend found that his colleague also has interest in writing. What if he and his colleague can find moments in the day to share their writing content?
    • Business: what business does your job supports? What’s the bigger picture of it? For example, are you in an IT development job? What social cause your company support? If it’s a public company, what is the sector of your company’s stock? How your work impacts the society? When we relate our work with a larger cause, it could help in bringing the engagement.
    • Technology/Skills: What skills do you need, to perform your current job? What are your areas of expertise? How valuable are those skills in the outside market? Is there any skill that other would like to learn from you? If you’re interested in mentoring others, would you like to mentor someone at work? Is there any particular skill you’d like to learn more at work?
  • Plan for periodic leaves. Sometimes, we feel that we have no choice other than going to the same work every week. Taking leaves helps us to reflect upon life. If it’s feasible, plan for a longer leave, like a week. If you’re free without any work tasks, what do you feel to do? For example, my one friend likes learning about technologies even when he’s out of the work. He’s naturally inclined towards technologies. By taking leaves, he realized that even if he doesn’t need a job for his financial needs, he’d like to write programs in his free time. He realized that it’s not the work type that he wants to change. Sometimes, he’s overwhelmed at work. So, the solution is to find out how can he prioritize his work so he can set the right expectations.
  • Plan for continuous evaluations: Plan for a periodic evaluation to assess where you’re. If you’re interested in something else, keep pivoting from the current situations. For example, after a decade of experience in a technical skills set, if you’re interested in moving to a new skills set, start planning for a steady move.

I’d love to know yours feedback on it. Thank you.

Some System Design basics

1 Point ContributorLeave a comment

Below are system design basics for software development:

  • How CDN works: CDN stands for Content Delivery Networks. CDN is a network of servers that distributes the content from original server to multiple locations by caching the content closest to the users’ locations.
  • Protecting passwords in a database: we should never store passwords directly into a database. These can be seen by database users. Also, these can be hacked. We should convert passwords into has values. To convert passwords into has values, we should use randomly generated salt values with each password. Salt is used to generate a unique hash.
  • Bare metal infrastructure: this is a term used for legacy physical server infrastructure. When an application needs the highest level of security, bare metals could be the most appropriate solution.
  • Virtual machines: This uses a hardware that is shared for multiple virtual servers. We use a hypervisor underneath guest OSs. The downside is that these could be vulnerable by noisy neighbor problems.
  • Containers: it’s a light weight stand alone package. We use a hardware and host OS. On top of it, a container engine is installed. On top of container engine, multiple containers are deployed. Containers are scalable and portable. Containers are less secured. They are vulnerable to security issues at OS level. To avoid security issues, we can run containers inside virtual machines.
  • How HTTP works: A user hits a URL on the browser. We use either http or https protocol. Second is a domain (like http://www.abc.com). We use a DNS (Domain Name Service) lookup to look for an IP for a domain. DNS information is generally cached. To look for a DNS, we have DNS servers. Finally a browser has the IP address of the server. Next, the browser get a TCP connection with the server. Browser sends a request to the server. Server sends an http response to the browser. Browser parses the responds and shows the response to the user on the browser.

References:

Software deployment strategies

1 Point ContributorLeave a comment

This is a part of system design knowledge base. In this article, we’ll discuss production code deployment strategies.

To deploy an application code, there could be multiple strategies, depending on a business need. Here are few common deployment strategies:

Full deployments:

Below two are full deployment strategies in that we deploy the compete code base or services into the production.

  • Multi-Service deployment: In this deployment strategy, we deploy all services within an environment (or a node). An example could be a single full deployment into a monolithic environment. The downside of this deployment is that it’s difficult to rollback.
  • Blue-Green deployment: In this deployment strategy, we keep a staging environment. Production code is moved to staging and new code is deployed to production. In case of the rollback situation, we can migrate codebase from staging to production.

Partial deployments:

Below two are partial deployment strategies in that we deploy the partial code base or services into the production. That means, we do not change entire services or codebase for all the users. Let’s refer to two types of partial deployments below:

  • Canary deployment: I also call it a pilot or a beta program deployment. In this deployment strategy, we release the new services to a smaller user base. Beta program could be an example of canary deployment.
  • A/B deployment: In this deployment strategy, we release new features to some users as an experimental release.
  • Difference in Canary and A/B deployments: The intent of canary deployment is mitigating the risks by releasing new services to some users. If some selected users are able to. use a new service as expected, we can release the service to rest all users. However, the intent of A/B testing is to experiment a new feature or a service with some users, to learn the success of a new feature or a service.

References:

Sources for product manager interviews

1 Point ContributorLeave a comment

Below are some useful links to prepare for product manager interviews:

As I learn more, I’ll update this page.