Circuit Breakers

To put it simply, circuit breaker in the context of software engineering is a pattern that prevents further damage made to a system/service when a section of it fails, allowing the other parts of the system to continue functioning smoothly. You would usually talk about the circuit breaker pattern when implementing resiliency and reliability in your system.

A circuit breaker detects failures and encapsulate the logic of handling those failures in a way that prevents the failure from constantly recurring. For example, they're useful when dealing with network calls to external services, databases, or really, any part of your system that might fail temporarily. By using a circuit breaker, you can prevent cascading failures, manage temporary errors, and maintain a stable and responsive system amidst a system breakdown.

Cascading Failures

Cascading failures occur when a failure in one part of the system triggers failures in other parts, leading to widespread disruption. An example is when a microservice in a distributed system becomes unresponsive, causing dependent services to timeout and eventually fail. Depending on the scale of the application, the impact of these failures can be catastrophic which is going to degrade performance and probably even impact user experience.

Circuit Breaker Patterns

A circuit breaker itself is a technique/pattern and there are three different states it operates which we will talk about:

1. Closed State: In a closed state, the circuit breaker allows all requests to pass through to the target service normally as they would. If the requests are successful, the circuit remains closed. However, if a certain threshold of failures is reached, the circuit transitions to the open state.

   Think of it like a fully operational service where users can log in and access data without issues. Everything is running smoothly.

   

2. Open State: In an open state, the circuit breaker immediately fails all incoming requests without attempting to contact the target service. The state is entered to prevent further overload of the failing service and give it time to recover. After a predefined timeout, the circuit breaker moves to the half-open state. A relatable example is this; Imagine an online store experiences a sudden issue where every purchase attempt fails. To avoid overwhelming the system, the store temporarily stops accepting any new purchase requests.

   

3. Half-Open State: In the half-open state, the circuit breaker allows a (configurable) limited number of test requests to pass through to the target service. And if these requests are successful, the circuit transitions back to the closed state. If they fail, the circuit returns to the open state. In the example of the online store i gave in the open state above, this is where the online store starts to allow a few purchase attempts to see if the issue has been fixed. If these few attempts succeed, the store will fully reopen its service to accept new purchase requests.

   This diagram shows when the circuit breaker tries to see if requests to Service B are successful and then it fails/breaks:

   

   The follow up diagram then shows when the test requests to Service B succeeds, the circuit is closed, and all further calls are routed to Service B again:

   

Note: Key configurations for a circuit breaker include the failure threshold (number of failures needed to open the circuit), the timeout for the open state, and the number of test requests in the half-open state.

Implementing Circuit Breakers in Go

It's important to mention that prior knowledge of Go is required to follow along in this article.

Let's continue.

As with any software engineering pattern, circuit breakers can be implemented in various languages. However, this article will focus on implementation in Golang. While there are several libraries available for this purpose, such as goresilience, go-resiliency, and gobreaker, we will specifically concentrate on using the gobreaker library.

Pro Tip: You can see the internal circuit breaker implementation of the gobreaker library here.

Let's consider a simple Go application where a circuit breaker is implemented to handle calls to an external API. This basic example demonstrates how to wrap an external API call with the circuit breaker technique:

Let's touch on a few important things:

1. gobreaker.NewCircuitBreaker function initializes the circuit breaker with our custom settings

2. cb.Execute method wraps the HTTP request, automatically managing the circuit state.

3. MaximumRequests is the maximum number of requests allowed to pass through when the state is half-open

4. Interval is the cyclic period of the closed state for the circuit breaker to clear the internal counts

5. Timeout is the duration before transitioning from open to half-open state.

6. ReadyToTrip is called with a copy of counts whenever a request fails in the closed state. If ReadyToTrip returns true, the circuit breaker will be placed into the open state. In our case here, it returns true if requests have failed more then three consecutive times.

7. OnStateChange is called whenever the state of the circuit breaker changes. You would usually want to collect the metrics of the state change here and report to any metrics collector of your choice.

Let's write some unit tests to verify our circuit breaker implementation. I will only be explaining the most critical unit tests to understand. You can check here for the full code.

1. We will write a test that simulates consecutive failed requests and checks if the circuit breaker trips to the open state. Essentially, after 3 failures, when the forth failure occurs, we expect the circuit breaker to trip (open) since our condition says counts.ConsecutiveFailures > 3 . Here's what the test looks like:

    t.Run("FailedRequests", func(t *testing.T) {
            // Override callExternalAPI to simulate failure
            callExternalAPI = func() (int, error) {
                return 0, errors.New("simulated failure")
            }
   
            for i := 0; i < 4; i++ {
                _, err := cb.Execute(func() (interface{}, error) {
                    return callExternalAPI()
                })
                if err == nil {
                    t.Fatalf("expected error, got none")
                }
            }
   
            if cb.State() != gobreaker.StateOpen {
                t.Fatalf("expected circuit breaker to be open, got %v", cb.State())
            }
        })
   

2. We will test the open > half-open > closed states. But we will first simulate an open circuit and call a timeout. After a timeout, we need to make at least one success request for the circuit to transition to half-open. After the half-open state, we need to make another success request for the circuit to be fully closed again. If for any reason, there's no record of a success request in the case, it will go back to being open. Here's how the test looks like:

    //Simulates the circuit breaker being open, 
    //wait for the defined timeout, 
    //then check if it closes again after a successful request.
        t.Run("RetryAfterTimeout", func(t *testing.T) {
            // Simulate circuit breaker opening
            callExternalAPI = func() (int, error) {
                return 0, errors.New("simulated failure")
            }
   
            for i := 0; i < 4; i++ {
                _, err := cb.Execute(func() (interface{}, error) {
                    return callExternalAPI()
                })
                if err == nil {
                    t.Fatalf("expected error, got none")
                }
            }
   
            if cb.State() != gobreaker.StateOpen {
                t.Fatalf("expected circuit breaker to be open, got %v", cb.State())
            }
   
            // Wait for timeout duration
            time.Sleep(settings.Timeout + 1*time.Second)
   
            //We expect that after the timeout period, 
            //the circuit breaker should transition to the half-open state. 
   
            // Restore original callExternalAPI to simulate success
            callExternalAPI = func() (int, error) {
                resp, err := http.Get(server.URL)
                if err != nil {
                    return 0, err
                }
                defer resp.Body.Close()
                return resp.StatusCode, nil
            }
   
            _, err := cb.Execute(func() (interface{}, error) {
                return callExternalAPI()
            })
            if err != nil {
                t.Fatalf("expected no error, got %v", err)
            }
   
            if cb.State() != gobreaker.StateHalfOpen {
                t.Fatalf("expected circuit breaker to be half-open, got %v", cb.State())
            }
   
            //After verifying the half-open state, another successful request is simulated to ensure the circuit breaker transitions back to the closed state.
            for i := 0; i < int(settings.MaxRequests); i++ {
                _, err = cb.Execute(func() (interface{}, error) {
                    return callExternalAPI()
                })
                if err != nil {
                    t.Fatalf("expected no error, got %v", err)
                }
            }
   
            if cb.State() != gobreaker.StateClosed {
                t.Fatalf("expected circuit breaker to be closed, got %v", cb.State())
            }
        })
   

3. Let's test the ReadyToTrip condition which triggers after 2 consecutive failure requests. We'll have a variable that tracks for consecutive failures. The ReadyToTrip callback is updated to check if the circuit breaker trips after 2 failures (counts.ConsecutiveFailures > 2). We will write a test that simulates failures and verifies the count and that the circuit breaker transitions to the open state after the specified number of failures.

    t.Run("ReadyToTrip", func(t *testing.T) {
            failures := 0
            settings.ReadyToTrip = func(counts gobreaker.Counts) bool {
                failures = int(counts.ConsecutiveFailures)
                return counts.ConsecutiveFailures > 2 // Trip after 2 failures
            }
   
            cb = gobreaker.NewCircuitBreaker(settings)
   
            // Simulate failures
            callExternalAPI = func() (int, error) {
                return 0, errors.New("simulated failure")
            }
            for i := 0; i < 3; i++ {
                _, err := cb.Execute(func() (interface{}, error) {
                    return callExternalAPI()
                })
                if err == nil {
                    t.Fatalf("expected error, got none")
                }
            }
   
            if failures != 3 {
                t.Fatalf("expected 3 consecutive failures, got %d", failures)
            }
            if cb.State() != gobreaker.StateOpen {
                t.Fatalf("expected circuit breaker to be open, got %v", cb.State())
            }
        })
   

Advanced Strategies

We can take it a step further by adding an exponential backoff strategy to our circuit breaker implementation. We will this article keep it simple and concise by demonstrating an example of the exponential backoff strategy. However, there are other advanced strategies for circuit breakers worth mentioning, such as load shedding, bulkheading, fallback mechanisms, context and cancellation. These strategies basically enhance the robustness and functionality of circuit breakers. Here's an example of using the exponential backoff strategy:

Exponential Backoff

Let's make a couple of things clear:

Custom Backoff Function: The exponentialBackoff function implements an exponential backoff strategy with a jitter. It basically calculates the backoff time based on the number of attempts, ensuring that the delay increases exponentially with each retry attempt.

Handling Retries: As you can see in the /api handler, the logic now includes a loop that attempts to call the external API up to a specified number of attempts (attempts := 5). After each failed attempt, we wait for a duration determined by the exponentialBackoff function before retrying.

Circuit Breaker Execution: The circuit breaker is used within the loop. If the external API call succeeds (err == nil), the loop breaks, and the successful result is returned. If all attempts fail, an HTTP 503 (Service Unavailable) error is returned.

Integrating custom backoff strategy in a circuit breaker implementation indeed aims to handle transient errors more gracefully. The increasing delays between retries help reduce the load on failing services, allowing them time to recover. As evident in our code above, our exponentialBackoff function was introduced to add delays between retries when calling an external API.

Additionally, we can integrate metrics and logging to monitor circuit breaker state changes using tools like Prometheus for real-time monitoring and alerting. Here's a simple example:

As you'll see, we have now done the following:

1. In L16-21, we define a prometheus counter vector to keep track of the number of requests and their state (success, failure, circuit breaker state changes).

2. In L25-26, the metrics defined are registered with Prometheus in the init function.

   Pro Tip: The init function in Go is used to initialize the state of a package before the main function or any other code in the package is executed. In this case, the init function registers the requestCount metric with Prometheus. And this essentially ensures that Prometheus is aware of this metric and can start collect data as soon as the application starts running.

3. We create the circuit breaker with custom settings, including the ReadyToTrip function that increases the failure counter and determines when to trip the circuit

4. OnStateChange to log state changes and increment the corresponding prometheus metric

5. We expose the Prometheus metrics at /metrics endpoint

Wrapping Up

To wrap up this article, i hope you saw how circuit breakers play a huge role in building resilient and reliable systems. By proactively preventing cascading failures, they fortify the reliability of microservices and distributed systems, ensuring a seamless user experience even in the face of adversity.

Keep in mind, any system designed for scalability must incorporate strategies to gracefully handle failures and swiftly recover. - Oluwafemi, 2024

What is the Beacon API?

The Beacon API is a JavaScript-based interface that allows web pages to send data to a server in the background, asynchronously, and without waiting for a response. As you would have guessed, this is useful for sending analytics or diagnostic information that doesn't typically require a response from your server or backend or just before the user leaves a page (for example, during the unload or beforeunload events).

Key Features of the Beacon API

• Asynchronous Data Transfer: Unlike AJAX requests, Beacon requests do not require a response from the server, allowing the user to navigate away from the page immediately without delay.

• Reliability: The data is transmitted to the server more reliably. Even if the page is being unloaded, the browser will attempt to send the Beacon data in the background.

• Efficiency: It uses HTTP POST requests and does not impact the performance or the loading time of the web page.

Let's get to it. How Does It Work?

The Beacon API's main function is navigator.sendBeacon(url, data), where url is the server endpoint to which data is sent, and data is the payload. The data can be any of several types, including ArrayBuffer, Blob, DOMString, FormData, or URLSearchParams as long as that's what your server or backend is expecting.

Implementation Steps

1. Check for Support: It's usually a good practice to first check if the user's browser supports it. If it does, then go ahead to use the beacon API

    if (navigator.sendBeacon) {
        // Beacon API is supported
    }
   

2. Sending Data: To send data with the Beacon API, we simply call navigator.sendBeacon() with the endpoint and data to be sent.

    window.addEventListener('unload', function(event) {
      var data = { userAction: 'pageExit', timestamp: Date.now() };
      var beaconUrl = 'https://example.com/analytics';
      navigator.sendBeacon(beaconUrl, JSON.stringify(data));
    });
   

3. Server-Side Handling: On the server, you'll receive the Beacon request just like any other POST request. The data can be processed or stored as needed.

Let's see more examples and uses cases

1. Sending Analytics Data on Page Unload: With the Beacon API, you can send user interaction data to an analytics endpoint when the user leaves the page. This is useful for capturing page session times, button clicks, or any actions the user performed on the page.

    window.addEventListener('beforeunload', function(event) {
      const analyticsData = {
        sessionDuration: Date.now() - window.sessionStartTime, // Assuming sessionStartTime was recorded at page load
        actions: window.userActions, // Assuming userActions were recorded during the session
      };
   
      navigator.sendBeacon('https://youranalyticsendpoint.com/data', JSON.stringify(analyticsData));
    });
   

2. Tracking Form Data Without Submission: The Beacon API can be used for draft logic to partially save form details. It allows you to send data to your backend asynchronously. It's useful for creating an autosave or draft feature in a web app.

   
    function saveDraft() {
      const formElement = document.getElementById('your-form-id');
      const formData = new FormData(formElement);
   
      // send draft data to the server using Beacon API
      const draftUrl = 'https://yourserver.com/saveDraft';
      const success = navigator.sendBeacon(draftUrl, formData);
      console.log('Draft save initiated:', success ? 'Success' : 'Failed');
    }
   
    // trigger the saveDraft function on form input (throttled)
    formElement.addEventListener('input', () => {
      // it's usually a best practice to use a throttle/debounce function to 
      //prevent too many Beacon requests
      if (window.draftSaveTimeout) {
        clearTimeout(window.draftSaveTimeout);
      }
   
      window.draftSaveTimeout = setTimeout(saveDraft, 500); // save draft every 500 ms of inactivity
    });
   
    // Additional save on page unload
    window.addEventListener('unload', saveDraft);
   

   When implementing this approach, It's important to also take note of the data size limitations and proper server-side handling, which also involves the backend acknowledging partial data and that it's expected that the backend handles these requests "silently"

Advantage Over Traditional Methods

Before the Beacon API, sending data to the server during unload events was less reliable. Traditional AJAX requests might be cancelled if they were initiated during these events, leading to data loss. The Beacon API ensures that the data is transmitted even after the page has started unloading.

Limitations and Considerations

• Since the Beacon API does not expect a response from the server, it's not suitable for tasks that require any response from the server.

• Some browsers may impose their own limits on the size of the data payload.

In general, the Beacon API provides a reliable, efficient method to send data to the server without affecting the user experience.

In the world of cloud computing, optimizing costs without sacrificing performance is a constant challenge. One way that Google Cloud offers to address this is through preemptible virtual machines.

For AWS folks, it's called AWS EC2 Spot Instance. The idea behind a spot instance and preemptible VM is the same.

Essentially, Preemptible VMs provide a cost-effective solution for running fault-tolerant and non-critical workloads. In this article, we will explore the benefits of preemptible VMs/Spot Instance, limitations, use cases, how the preemptible process work, creating a preemptible VM from a regular instance and many more.

What exactly is a Preemptible VM?

Google Cloud preemptible VMs are similar to regular instances but come with a significant cost advantage. Well, the tradeoff is that these VMs/Spot Instances may be terminated by Google/AWS at any time albeit with a 30secs/2 mins notice. While this means they are not suitable for long-running, critical tasks, they are ideal for batch processing, distributed computing, and fault-tolerant applications.

Before we move ahead to spin up some preemptible VM instances (actually, in a followup article), i'd like to highlight the benefits, limitations and use cases.

Benefits of Preemptible VMs:

1. Cost Efficiency: Preemptible VMs are priced significantly lower than regular instances, providing cost savings of up to 80%. This makes them an attractive option for workloads that can tolerate occasional interruptions.

2. Scalability: By leveraging preemptible VMs, you can easily scale your infrastructure at a fraction of the cost. This is particularly advantageous for bursty (occurring at intervals in a short timespan) workloads that require additional resources temporarily.

3. High Availability: Preemptible VMs can be used in combination with managed instance groups and autoscaling to ensure high availability and fault tolerance. The system automatically replaces preempted VMs with new ones, maintaining the desired level of capacity.

Limitations of Preemptible VMs

1. Limited Availability: Preemptible VMs are available on a "best-effort" basis and their availability is not guaranteed. They are offered at a significantly reduced price compared to regular VMs because Google Cloud can terminate them at any time. So, this means that they may not be suitable for applications requiring strict uptime or critical workloads.

2. Maximum Runtime: Preemptible VMs have a maximum runtime limit of 24 hours. After this time, they will be automatically terminated by Google Cloud. If your application or job requires longer execution times, you need to account for this limitation and design your solution accordingly :).

3. Termination without Warning: Preemptible VMs can be terminated at any time, without any advanced warning. While Google Cloud typically provides a 30-second notification before termination, your applications and processes must be designed to handle sudden interruptions and gracefully recover or resume operations when a VM is preempted.

4. Limited Quantity: There is a finite capacity of preemptible VMs available within a specific region and zone. If the demand for preemptible VMs exceeds the available capacity, you may not be able to launch new instances until capacity becomes available.

5. Resource Constraints: Preemptible VMs have some resource constraints compared to regular VMs. For example, they cannot be live migrated to other hosts, and they have a limited amount of CPU and memory resources. These constraints may impact certain workloads or applications that require specific configurations or resource-intensive operations.

Despite these limitations, preemptible VMs can still be a cost-effective option for certain use cases which are mentioned below:

Usecases of Preemptible VMs

1. Batch Processing: Preemptible VMs are ideal for batch processing workloads that can be divided into smaller tasks or jobs. You can leverage the significant cost savings offered by preemptible VMs to run large-scale data processing, ETL or other batch jobs. If a batch job is preempted, it will be restarted on a new preemptible VM. However, the job may lose some of its state, so it is important to design the job in a way that minimizes the impact of preemptions.

   For example, let's say you have a batch job that is processing a large file or running image analysis or machine learning training, you could have a checkpoint logic that checks its progress every few minutes or seconds (the latter would be recommended) configurable by you and saves the state so that if the job is preempted at any point, it can continue processing from the last saved checkpoint when another VM is up and running - this is called checkpointing technique.

2. Test and Development Environments: Preemptible VMs can be used for creating temporary or short-term test and development environments. For instance, if your dev team requires isolated environments for testing, experimenting, or prototyping, preemptible VMs can provide the necessary resources at a much lower cost.

3. Non-Critical Workloads: Applications or workloads that can tolerate occasional interruptions or delays are good candidates for preemptible VMs. Examples include non-production environments, non-critical background tasks, non-time-sensitive data processing, or non-mission-critical applications.

4. DevOps: Preemptible VMs can be used for DevOps tasks, such as running continuous integration and continuous delivery (CI/CD) pipelines. These tasks can be interrupted and restarted without any loss of data, so they are well-suited for preemptible VMs. By leveraging the cost savings, you can scale your CI/CD infrastructure without incurring high expenses during idle or low-demand periods.

5. High-Performance Computing (HPC): For certain HPC workloads, preemptible VMs can be used to increase compute capacity while managing costs. Tasks such as rendering, simulation, scientific calculations, or distributed computing can benefit from the availability of preemptible VMs.

6. Web Crawlers or Scrapers: Preemptible VMs can be used for web crawling or scraping tasks where the workload can be divided into smaller chunks or parallelized. The lower costs associated with preemptible VMs make them an attractive option for scraping data from websites or conducting periodic web crawls.

Overall, batch jobs can be run on Google Cloud preemptible VMs, but it is important to design the job in a way that minimizes the impact of preemptions. By following these tips, you can save money on your batch processing jobs without sacrificing reliability. It is important to assess your application's requirements, resilience, and cost considerations before incorporating preemptible VMs into your infrastructure.

💡Spot VMs are the latest version of preemptible VMs. New and existing preemptible VMs continue to be supported, and preemptible VMs use the same pricing model as Spot VMs. However, Spot VMs provide new features that preemptible VMs do not support. For example, preemptible VMs can only run for up to 24 hours at a time, but Spot VMs do not have a maximum runtime unless you limit the runtime. You can read more on them and decide which one to use for your project and/or tasks.

Preemption Process

According to Google Cloud documentation, the preemption process is as follows:

1. Once Compute Engine needs the capacity, Google sends a preemption notification as an Advanced Configuration and Power Interface (ACPI) G2 Soft Off signal -- a standard motherboard soft shutdown command, which every OS can handle -- that signals the system must reboot.

2. Ideally, the Soft Off signal then triggers a shutdown script that users have previously configured to save any system state and application data, terminate processes and stop the VM.

3. If the instance is still running after 30 seconds, GCE sends an ACPI G3 Mechanical Off signal to the OS, which is the equivalent of pulling the power on a server.

4. The Compute Engine instance then enters a terminated state, which preserves its configuration settings, metadata and attachments to other resources -- such as storage volumes -- but destroys in-memory data and VM state. Users can choose to restart or delete an instance in a terminated state, or leave it terminated indefinitely

Preempted instances still appear in your project, but you are not charged for the instance hours while it remains in a TERMINATED state.

Converting a regular VM into a preemtible VM

There's no direct way to convert an existing regular VM into a preemtible VM but there's a workaround and i'll be showing you the steps.

Step 1:

Goto snapshots page here and click on Create Snapshot.

Step 2:

Input the name of your snapshot, click on "source disk" to choose which VM instance you want to create a snapshot from and then click on "Create"

Step 3:

Once a snapshot is created, click on it to view details and then click on "Create Instance".

Step 4:

Scroll to the near bottom under Available policies. Standard is selected by default but you have to select "Spot" because that's what we want to create. You will also notice that the price when you choose Standard is very different from (and higher than) the Spot and that's because it's a preemtible VM. Once that's done, just click on Create.

Step 5: That's it. You've successfully created a preemptible VM instance from a regular VM!

ICYMI: What to keep in mind when using Preemtible VMs

1. Your application must be fault-tolerant: Your application must be able to handle being interrupted and restarted. If your application cannot handle being interrupted, then you should not use preemptible VMs.

2. Your application must be stateless: Your application must not store any state on the VM. If your application stores state on the VM, then it will be lost when the VM is preempted.

3. Your application must be able to run quickly: Your application should be able to complete its work within 24 hours, wrap up and save the current state within the 30-seconds notice period. If your application takes longer than that to run, then it is possible that it will be preempted before it completes.

Conclusion

Google Cloud preemptible VMs offer an excellent opportunity to optimize costs while leveraging the power of cloud computing. By understanding the benefits, limitations and implementing them in your applications, you can unlock significant savings and scalability. However, it's important to carefully assess the suitability of preemptible VMs for your specific use case and ensure appropriate fault tolerance measures are in place.

Efficient task management is vital in modern distributed and scalable cloud environments. Google Cloud Tasks offers a managed solution that simplifies the distribution and execution of tasks across various components of your application. In this article, we will explore the key features of Google Cloud Tasks and demonstrate how to leverage them using Node.js code snippets.

What is Google Cloud Tasks?

Google Cloud Tasks is a fully managed task distribution service that allows you to reliably enqueue and execute tasks. It provides features such as task queuing, scheduling, retries, and prioritization, making it an ideal choice for building scalable and responsive applications.

Getting Started with Google Cloud Tasks

To start using Google Cloud Tasks, follow these steps:

Step 1: Enable the Cloud Tasks API

Ensure that you have enabled the Cloud Tasks API in your Google Cloud project. You can do this through the Google Cloud Console or by using the gcloud command-line tool.

gcloud services enable cloudtasks.googleapis.com

Step 2: Create a Task Queue

A task queue is a container for your tasks. Create a task queue by specifying a name and other optional parameters such as maximum task attempts, rate limits, and worker constraints.

Step 3: Enqueue Tasks:

Enqueue tasks to the task queue by specifying the request method, URL, body and any other optional parameters that fit your needs. The payload can contain any data necessary for task execution. The body needs to be in base64 in order for the data to be sizeable and easily transmitted over the network.

Step 4: Task Handler:

Implement a task handler that processes the tasks. This could be a separate route or function that receives the tasks, extracts the payload, and performs the necessary actions.

Note that it is very important to return a 200. Any status code other than that indicates that the execution failed and Cloud Task will keep on retrying (depending on your queue configurations).

That's it, basically.

If you want to get information about a task and/or delete a task, you can use the methods below.

Let's go through a simple example of how to use the cloud task functions we created. There are different use cases of Cloud Tasks (i mentioned them in the later part of this article) but for the sake of simplicity of this article, let's imagine that we have to do different batch jobs, sort of like a sequence of jobs.

We can have a BatchJobService that has functions which call the cloud task functions that we created.

import { ITaskService } from "./TaskService";

const baseUrl = "http://whatever-your-base-url-is";

export interface IObject {
    [key: string]: any;
}

export class BatchService {

    constructor (private taskService: ITaskService) {}

    async createFirstBatchTask() {
        //create queue
        const queueName = await this.taskService.createTaskQueue('first-batch-queue');

        //add a task to the queue you created above
        await this.taskService.createTask(queueName, {
            taskName: "first-batch-task",
            url: `${baseUrl}/create-first-batch`,
            data: { //whatever data you want to send or pass
                operationType: "batch",
                value: 20
            }
        });
    }

    async processFirstBatchTask(data: IObject) {
        console.log(data); //{operationType: "batch", value:20}
    }
}

The example above is pretty explanatory and there's not much to talk about. The first function createFirstBatchTask creates a task queue (like a container), and then enqueues a task into the queue while passing the data to be sent, url to process the data and what HTTP method to use.

The second function processFirstBatchTask is the handler which processes whatever data and does whatever it wants with it.

The full code can be seen here: https://github.com/SirPhemmiey/cloud-task-tutorial

If you have used Google Cloud Pub/Sub before, you'd probably be wondering about the difference between Cloud Pub/Sub and Cloud Tasks, just like I did before i started using Cloud Tasks. Truth is, they are both powerful services provided by GCP, but they serve different purposes and have distinct characteristics.

Amongst other differences between the two, the core difference is in their message handling and invocation; implicitly and explicitly.

What does Implicit and Explicit Invocation even mean?

Implicit: In this case, the publisher has no control over the delivery of the message. Pub/Sub aims to decouple publishers of events and subscribers to those events. Publishers do not need to know anything about their subscribers.

Explicit: By contrast, Cloud Tasks is aimed at explicit invocation where the publisher retains full control of execution. The publisher can tell how the message should be delivered, when the message should be delivered and what to pass in the message. Full control.

Another benefit of Cloud Tasks is you can pause/resume the queue using Cloud Console and CLI command to stop/start the processing of tasks, very similar to Google Cloud Scheduler.

1. Detailed Comparison of Cloud Tasks and Pub/Sub

   

   Advanced Features and Use Cases of Cloud Tasks

   Google Cloud Tasks offers several advanced features and use cases:

   1. Task Scheduling: You can schedule tasks to be executed at specific times or intervals. Set the scheduled time when enqueuing the task, and Google Cloud Tasks ensures the task is executed accordingly. For instance, you may want to send an email 1 month after sign-up for the trial period. Without using Cloud Tasks, you'd ideally have a cron job to check the difference between dates and send the email. But Cloud Tasks saves you this query!

   2. Task Retries and Acknowledgment: Google Cloud Tasks automatically retries failed tasks based on configurable settings. Tasks can also be acknowledged upon completion, allowing you to track their status and handle any failures or retries.

   3. Ordering and Prioritization: You can control the order in which tasks are executed by specifying task priorities. Higher-priority tasks are processed before lower-priority ones, ensuring important tasks are handled promptly.

   4. Monitoring and Insights: Google Cloud Tasks provides visibility into task execution with built-in monitoring and logging. You can access metrics, logs, and error information to track the performance and health of your task processing.

   5. Point-2-Point Communication: Asynchronous call between 2 microservices.

   6. Control Traffic: Need to control the rate so that worker's scalability is under control. e.g. Push an asynchronous image processing job, hitting an API with max requests etc

Limitations of Google Cloud Tasks

As much as Google Cloud Tasks helps with efficient task management on the cloud, it does have limitations, some of which i don't like and i wish that the limitation is removed in the near future. There are a couple of limitations but i'll highlight the "most important ones" to know and keep in mind

1. Limited task payload size: Google Cloud Tasks imposes a limit on the size of the task payload, which is currently set at 1MB. So, if your tasks require larger payloads, you may need to consider alternative solutions or split the payload across multiple tasks.

2. Task retention period: Tasks in Google Cloud Tasks have a limited retention period, which is currently set at 31 days. This means that any task added to a queue must be executed within 31 days. If a task is not processed within this period, it will be automatically deleted. So, you need to ensure your tasks are processed in a timely manner to avoid losing any important data.

3. Task execution time limits: Google Cloud Tasks imposes a maximum execution time limit for tasks, which is currently set at 10 minutes. If your tasks require longer execution times, you'll need to consider other mechanisms or split the work into multiple tasks.

4. Queue Recreation: If you delete a queue, you must wait for 7 days before creating a queue with the name again. One of the limitations i dislike because this makes me rethink about naming my queues carefully.

5. Queue dispatch rate: This refers to the maximum rate at which tasks can be dispatched from a queue. The limitation is that you can only dispatch 500 taks in a queue per second. So, if you want to dispatch more than that, it's best to use multiple queues.

6. Task de-duplication window: As much as you can create multiple tasks with different names in queue, once a task is deleted, you'll have to wait for about 1 hour to use the same name again.

7. Maximum schedule time for a task: This is the maximum amount of time in the future that a task can be scheduled. If you want to schedule a task to be ran more than 30 days from the current date, it's going to throw an error. This is arguably the limitation i dislike the most.

   It's important to consider these limitations when evaluating Google Cloud Tasks for your specific use case. While it is a powerful task queuing service, understanding its constraints will help you make informed decisions and plan accordingly for your application requirements.

Conclusion

Google Cloud Tasks simplifies the management of distributed tasks in your applications. Its powerful features, such as task queuing, scheduling, retries, and prioritization, make it an excellent choice for building scalable and reliable systems. In this article, we covered the basics of using Google Cloud Tasks and demonstrated how to create task queues, enqueue tasks, and handle them using a task handler in Node.js. We talked about advanced features, and use cases of Cloud Tasks to help you make an informed decision. We also talked about the differences between Cloud Pub/Sub and Cloud Tasks. By leveraging Google Cloud Tasks, you can focus on your application's business logic while relying on a fully managed service to handle task distribution and execution efficiently.

Reference

https://medium.com/google-cloud/cloud-tasks-or-pub-sub-8dcca67e2f7a

Prerequisites:

• A GCP account with a project and a VM instance.

• Ansible is installed on your local machine.

• An SSH key pair to access the VM instance.

• You must have followed my previous article here because you will be needing to modify the playbook that provisions a VM.

Step 1:

Create an inventory file Create a file named inventory and add the IP address or hostname of the VM instance you want to install Redis on. For example:

[redis]
<ip address> or <hostname>

Step 2:

Create a playbook Create a file named redis-playbook.yml and add the following tasks:

---
- name: Redis Installation
  hosts: redis
  become: true

  tasks:
    - name: Update package repositories
      yum:
       update_cache: yes

    - name: Install Redis
      yum:
        name: redis
        state: present

    - name: Start Redis service
      systemd:
        name: redis
        state: started
        enabled: yes

This playbook uses the yum module to update the package repositories on the target system and it also installs the Redis package. The state parameter is set to present to ensure that the package is installed if it is not already present.

After that, the Redis service is started with the systemd module.

Step 3:

Run the playbook using the following command:

ansible-playbook redis-playbook.yml -i inventory --private-key=/path/to/ssh/key

This command runs the redis-playbook.yml playbook on the hosts specified in the inventory file and uses the SSH key specified in --private-key to access the VM instance.

Step 4:

Verify that Redis is installed and running by connecting to the VM instance using SSH and running the following command:

redis-cli ping

TIP: You can use the command to ssh into your instance

ssh <external-ip-address> -i path/to/private/key

If Redis is running, the command should return PONG.

Now that we've confirmed that Redis is running in the VM instance, we'll need to connect to it outside the instance. Right now, if we try to do it, we're going to get an error (or it will most likely timeout). So this means we need to enable remote access to Redis and also update our firewall to accept TCP connections on port 6379.

You first need to install the community versions of Ansible's firewall and google modules with the command:

ansible-galaxy collection install community.general community.google

Step 5:

You will need to add apache-libcloud to the list of requirements in requirements.yml

 pip_package_requirements:
       ...
      - "apache-libcloud"

Once that is successful then copy the following tasks:

- name: Configure Redis
  become: yes
  lineinfile:
    path: /etc/redis/redis.conf #or /etc/redis.conf if you get an error that /etc/redis/redis.conf does not exist
    regexp: "{{ item.regexp }}"
    line: "{{ item.line }}"
  with_items:
    - { regexp: "^bind .*", line: "bind 0.0.0.0" }
    - { regexp: "^port .*", line: "port 6379" }
    - { regexp: "^# requirepass .*", line: "requirepass your_password_here" }
  notify: Restart Redis service

 - name: Allow incoming connections on port 6379
   community.general.ufw:
     rule: allow
     port: 6379
     proto: tcp

 - name: Reload firewall rules
   community.general.ufw:
     state: enabled

The task above updates the Redis configuration file and you'll notice that we also set a password. Remember to replace your_password_here with your preferred password.

This is the full Ansible playbook for Redis configuration:

Step 6:

We will also need to create a GCP compute firewall by adding the following tasks in the playbook we created in my previous article here.

Copy the following tasks into your playbook. Be careful of indentation though :)

  - name: Create firewall policy for Redis
      gcp_compute_firewall:
        name: "{{ firewall_policy_name }}"
        priority: 1000
        direction: "INGRESS"
        project: "{{ gcp_project }}"
        service_account_file: "{{ gcp_cred_file }}"
        auth_kind: "{{ gcp_cred_kind }}"
        allowed:
        - ip_protocol: "tcp"
          ports:
            - 6379
        target_tags:
          - "redis"
        state: present
      register: firewall_policy_result
      #when: firewall_policy_result is not defined

    - name: Print firewall_policy_result
      debug:
        var: firewall_policy_result

    - name: Add firewall policy to Redis instance
      community.google.gce_tag:
        instance_name: "{{ instance_name }}"
        tags: redis
        zone:  "{{ zone }}"
        project_id: "{{ gcp_project }}"
        state: present

In the tasks above, we're creating a GCP compute firewall policy and we're assigning the policy to our Redis instance.

This is the full Ansible playbook for VM provisioning and firewall policy creation:

The content of requirements.yml is this:

It's important to note that you'll have to grant your service account permission to create/manage a firewall by assigning the Compute Network Admin role in GCP's IAM page here.

Step 7:

Verify the remote connection by connecting remotely to the Redis instance with the command:

redis-cli -h <ip> -p <port> -a <password> ping

And you should be able to connect and get a PONG.

Congratulations! You have successfully installed Redis on a GCP VM instance using Ansible. You can now use Redis as a database, cache, or message broker in your application.

I know this must have been a lot. And that's because this is the first time. Trust me, this can save you and anyone else hours of time because with just this file, you can repeat your tasks and you can deploy the same configuration to multiple VM instances at once.

Conclusion

In this article, we learned how to install Redis on a GCP VM instance using Ansible. We also learned how to configure Redis by updating the configuration file, allow incoming connections, create firewall rules and policy. Ansible provides a simple and efficient way to automate the deployment and configuration of Redis on GCP VM instances.

Thank you for reading my article on Simplifying Your Redis Deployment with Ansible! Stay tuned for my upcoming articles on adding monitoring to your Redis server and the many advantages of creating a Redis cluster using Ansible. Don't miss out on the benefits of high availability, scalability, and fault tolerance that a Redis cluster can provide for your applications.

What is Ansible?

Ansible is an open-source automation tool that helps with configuration management, application deployment, and task automation. It uses a simple, human-readable language to describe automation tasks and is easy to use even for those without a programming background. Ansible is agentless, which means that it doesn’t require software to be installed on the target host to manage it.

Why use Ansible for infrastructure management?

Ansible can help streamline infrastructure management in several ways:

1. Consistency: Ansible ensures that infrastructure is provisioned and configured in a consistent manner across all hosts. This can help reduce errors and make troubleshooting easier.

2. Scalability: Ansible can manage large-scale deployments with ease, making it an ideal choice for organizations with a significant number of hosts.

3. Reusability: Ansible’s modules and playbooks can be reused across different projects and environments, making it a valuable asset for organizations that require flexibility and agility.

4. Time-saving: Ansible’s automation capabilities can significantly reduce the time and effort required to manage infrastructure, freeing up IT teams to focus on more strategic initiatives.

Another benefit of using Ansible for infrastructure management is the ability to use it across different cloud providers and even on-premises infrastructure. This means that you can use the same automation tool to manage infrastructure across different environments, reducing the need for specialized skills and tools.

Prerequisites

To follow this article and use Ansible to provision Google Cloud VMs, you should have some basic knowledge of the following:

1. Linux: Ansible is primarily a Linux automation tool, so you should have some familiarity with Linux commands, file systems, and permissions.

2. Cloud Computing: You should have a basic understanding of cloud computing concepts, such as virtual machines, cloud providers, and cloud infrastructure.

3. Google Cloud Platform (GCP): You should have a GCP account and some familiarity with the GCP console, including creating and managing VMs.

4. Ansible: You should have a basic understanding of Ansible concepts, such as playbooks, modules, variables, and tasks.

If you are not familiar with any of these concepts, you may want to spend some time learning about them before attempting to follow this article. Many online resources are available for learning about Linux, cloud computing, GCP, and Ansible.

Fret not though, we won’t be going very deep into them, and I’ll be guiding you through the most important concepts.

Ansible can provision and automate anything on GCP and other cloud providers. It’s not limited to provisioning VMs on GCP only.

Long story short, let’s dive straight into what you’re here for!

Provisioning GCP VMs with Ansible

To provision GCP VMs with Ansible, you’ll first need to install ansible on your machine by following the instructions here. We will also need to install the Ansible GCP module. Check here to see a list of GCP collections. This module allows you to interact with the GCP API and perform tasks such as creating, starting, stopping, and deleting VMs.

Instead of just installing a single module, it is better to install the whole google cloud collection to avoid getting any errors that something is missing. Here’s the command:

//install ansible if you don't have it
pip install ansible

//install google cloud ansible collection
ansible-galaxy collection install google.cloud

Before you can use the Ansible GCP module though, you’ll need to set up a service account and download the service account key in JSON format (you’ll need the path to it later).

Now you’re ready to create a playbook. A playbook is a file that describes a set of tasks to be executed on a group of hosts. In this case, we’ll create a playbook that provisions a Google Cloud VM. Copy and paste the content of this file to your machine. The filename is playbook.yml

This playbook creates a VM instance with the specified image, machine type, disk size and type, network, and tags.

You can execute the playbook with the ansible-playbook command like this:

ansible-playbook initial.yml

It’s that simple! You should get a response in your terminal like this:

And when I go to the VM instance in the google cloud console here, I can see it created and running!

That’s simple, right? I bet it is! You can see that it took us less than a minute to do this. It may interest you to know that you can run this playbook many times, which means the process is Idempotent. Instead of hard coding the values, you can also pass them as an argument to the playbook. You may want to look into the documentation on how to do that.

Let’s talk briefly about some lines in the file:

Line 6: This is how you set local variables in an Ansible yml file and they can be accessed throughout the file.

Line 7–8: You’ll need to input the right credentials.

Line 11: Instance name can be anything. But ideally, it’s recommended to make it meaningful.

Line 12–13: Zone and region can be any acceptable zone and region respectively. To see a list of all available zones and regions, run the command:

//list available regions
gcloud compute regions list --project=<project-id>

//list available zones
gcloud compute zones list --project=<project-id>

Line 14–15: That’s my preferred machine type and machine image. To see the available list of images, run the command:

gcloud compute images list --uri --project=<project-id>

My Stackoverflow answer here might help you. Please upvote if it helped you 😢.

Line 25: Using the register key, we’re saving the result of the task into that variable. This is useful if we want to perform another task based on the result of a previous task.

Line 48: Finally, we use the debug module to print the VM's IP address. You can see how we used gcp_ip.address to show the address.

NOTE: If by any chance you get an error that a package or library doesn’t exist, you can just run the command:

pip3 install <package name>

I know this is just provisioning a VM and doing nothing with it. In my upcoming articles, I’ll take you through installing and configuring additional things on the VM to make it useful all with Ansible. Stay tuned.

Conclusion

Automating infrastructure management tasks with Ansible can greatly improve efficiency and reduce errors. Provisioning and managing GCP VMs with Ansible is a powerful way to streamline infrastructure management and ensure that your systems are always configured to your specifications. Whether you’re deploying a new application, scaling up an existing system, or just need to make updates to your infrastructure, Ansible provides a simple and powerful way to automate these tasks. By following the steps outlined in this article, you can start using Ansible to provision Google Cloud VMs in no time.

Of course, this is just the beginning of what you can do with Ansible and Google Cloud. Ansible has a wide range of modules for managing different aspects of cloud infrastructure, from networking to security to storage. You can use Ansible to automate the deployment of applications, configure load balancers, and much more.

If you liked this article, please leave a clap or even a comment and don’t forget to follow me to get updated when I publish another one. Thanks!

Google Cloud Run is a serverless container platform that enables developers to run applications in a fully managed environment. It allows you to deploy stateless containers on a pay-as-you-go basis and auto-scales your application based on incoming traffic.

What is Github Actions?

GitHub Actions is a powerful workflow automation tool that allows developers to automate their development workflows. It integrates well with Google Cloud Run, making it easy to deploy applications from GitHub to Cloud Run.

In this article, I will be deploying a containerized web application to Google Cloud Run using GitHub Actions.

NOTE: If you want to use Gitlab CI/CD instead of GitHub Actions, see my other article here.

Let’s continue….

Prerequisites

Before we dive into the tutorial, make sure you have the following:

• A Google Cloud Platform account

• A GitHub account

• Docker installed on your local machine

Step 1: Set up your project on Google Cloud

Before we can deploy our application to Google Cloud Run, we need to create a new project on Google Cloud Platform and enable the Cloud Run API. Here’s how to do it:

1. Go to the Google Cloud Console.

2. Click on the project dropdown menu and select “New Project”.

3. Give your project a name and click “Create”.

4. Once your project is created, click on the “Activate Cloud Shell” button on the top right corner of the page.

5. Run the following command to enable the Cloud Run API:

gcloud services enable run.googleapis.com

An alternative way to enable Cloud Run API

1. Go to the Google Cloud Console and select your project.

2. In the left navigation menu, click on “APIs & Services” and then “Dashboard.”

3. Click on the “+ ENABLE APIS AND SERVICES” button.

4. Search for “Cloud Run API” and click on it.

5. Click the “Enable” button.Step 2: Create a Dockerfile

Next, we need to create a Dockerfile for our application. This file will contain instructions on how to build a container image for our application.

Here’s an example Dockerfile for a Node.js application:

# Use the official Node.js image
FROM node:14-alpine

# Set the working directory
WORKDIR /app

# Copy the package.json and package-lock.json files
COPY package*.json ./

# Install the dependencies
RUN npm install --production

# Copy the rest of the application code
COPY . .

# Expose port 8080
EXPOSE 8080

# Start the application
CMD ["npm", "start"]

Save this file in the root directory of your project.

Step 3: Build and test the container locally

Before deploying our container to Google Cloud Run, let’s build and test it locally. Run the following command to build the container image:

docker build -t <your-image-name> .

Replace <your-image-name> with a name for your container image. Once the build is complete, run the container with the following command:

docker run -p 8080:8080 <your-image-name>

This will start the container and map port 8080 on your local machine to port 8080 inside the container. Open your web browser and go to http://localhost:8080 to test your application.

Step 4: Set up GitHub Actions

GitHub Actions is a powerful tool that allows you to automate your software development workflows. In this step, we will be creating a GitHub Actions workflow to build and deploy our container to Google Cloud Run.

1. In your GitHub repository, click on the “Actions” tab.

2. Click on the “Set up a workflow yourself” button.

3. Replace the contents of the file with the following code:

name: "Deploy to Google Cloud Run"

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Google Cloud SDK
        uses: google-github-actions/setup-gcloud@master
        with:
          project_id: <your-project-id>
          service_account_key: ${{ secrets.GCP_SA_KEY }}
          export_default_credentials: true

      - name: Configure docker for GCP
        run: gcloud auth configure-docker

      - name: Build and push Docker image
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: gcr.io/<your-project-id>/<your-image-name>:latest
          build-args: |
            HTTP_PORT=8080

      - name: Deploy to Cloud Run
        uses: google-github-actions/deploy-cloudrun@main
        with:
          image: gcr.io/<your-project-id>/<your-image-name>:latest
          service: <your-service-name>
          region: <your-region>
          platform: managed
          allow-unauthenticated: true
          env_vars: |
              FOO=bar
              ZIP=zap

Replace <your-project-id>, <your-image-name>, <your-service-name>, and <your-region> with your own values.

You can see more here on how to use the google cloud run github actions.

4. Click on the “Start commit” button and commit the changes to the repository.

Step 5: Deploy to Google Cloud Run

Once the GitHub Actions workflow completes successfully, your container should be deployed to Google Cloud Run. To verify that your application is running, go to the Google Cloud Console, select your project, and click on “Cloud Run” in the sidebar. You should see your service listed there.

Click on the service to view its details, including the URL for your application. Open this URL in your web browser to test your deployed application.

Congratulations! You have successfully deployed a containerized web application to Google Cloud Run using GitHub Actions.

If you liked this article, please leave a clap or even a comment and don’t forget to follow me to get updated when I publish another one. Thanks!

In this article, I will walk you through the process of setting up a GitLab CI/CD pipeline to deploy your code to Google Cloud Run.

NOTE: If you want to use Github Actions instead of Gitlab CI/CD, see my other article here.

Let’s continue….

Prerequisites

Before we get started, make sure that you have the following:

• A Google Cloud account

• A GitLab account with a repository containing your code

• The Google Cloud SDK installed on your local machine

• Docker installed on your local machine

Step 1: Create a Google Cloud Run Service

First, we need to create a Google Cloud Run service that will host our application. To do this, follow these steps:

1. Open the Google Cloud Console and navigate to the Cloud Run page.

2. Click the “+ Create Service” button.

3. Choose your preferred region and select the “Deploy one revision from an existing container image” option.

4. Enter a name for your service and select the container image you want to deploy.

5. Click “Create” to create your Cloud Run service.

Step 2: Authenticate the Google Cloud SDK

To deploy your code to Cloud Run, you need to authenticate the Google Cloud SDK on your local machine. To do this, follow these steps:

1. Open your terminal and run the following command:

gcloud auth login

2. Follow the prompts to log in to your Google Cloud account.

Step 3: Create a GitLab CI/CD Pipeline

Now that we have our Cloud Run service set up and authenticated the Google Cloud SDK, we can create a GitLab CI/CD pipeline to automate our deployment process.

1. In your GitLab repository, create a new file called .gitlab-ci.yml.

2. Add the following code to the file:

    image: docker:latest
   
    services:
      - docker:dind
   
    before_script:
      - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
   
    deploy:
      image: google/cloud-sdk:latest
      script:
        - gcloud auth activate-service-account --key-file=google-creds.json
        - gcloud config set project $PROJECT_ID
        - gcloud builds submit --tag gcr.io/$PROJECT_ID/$CI_PROJECT_NAME:$CI_COMMIT_SHA
        - gcloud run deploy --image=gcr.io/$PROJECT_ID/$CI_PROJECT_NAME:$CI_COMMIT_SHA --platform=managed --region=$CLOUD_RUN_REGION --allow-unauthenticated --update-env-vars=VAR1=value1,VAR2=value2 --quiet
      only:
        - master
   

3. Replace $PROJECT_ID with your Google Cloud project ID and $CLOUD_RUN_REGION with your preferred region.

4. Add any environment variables you need to the --update-env-vars flag.

5. Commit and push your changes to your GitLab repository.

Step 4: Configure GitLab CI/CD Variables

Finally, we need to configure some variables in GitLab CI/CD to authenticate our Google Cloud account and registry. To do this, follow these steps:

1. In your GitLab repository, navigate to “Settings” > “CI/CD” > “Variables”.

2. Add the following variables:

• GOOGLE_APPLICATION_CREDENTIALS - the contents of your Google Cloud service account key file.

• PROJECT_ID - your Google Cloud project ID.

• CI_REGISTRY_USER - your GitLab username.

• CI_REGISTRY_PASSWORD - your GitLab personal access token.

Conclusion

Congratulations! You now have a fully automated GitLab CI/CD pipeline that deploys your code to Google Cloud Run. With this setup, you can focus on writing code and let GitLab and Google Cloud handle the rest.