I recently hosted a panel with AWS, Amgen, Ardigen, and Seqera on running omics workflows in the cloud. 


Here were my key takeaways from the talk: 

You can watch the full video here: 

<iframe width="560" height="315" src="https://www.youtube.com/embed/zQyh7LF6_h4?si=4QVkjvxM0E6Kj5Oq" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

## Running omics workflows on the cloud is about understanding what tradeoffs you’re making. 

<iframe width="560" height="315" src="https://www.youtube.com/embed/T_keBXj_iuQ?si=KhA4UWL9n5647YkQ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

“For example if you're running a production pipeline where you just need to keep it churning, but you don't need to have the results immediately, you can wait and deal with retries. You might also not necessarily need the latest and greatest hardware. 

However, if you're in an R&D activity, the time of your developers is more expensive than the time of the machines. 

So you need to be able to have your developers progress very rapidly, be able to iterate extremely rapidly, because the cost you're paying the humans who are operating machines is higher, and also the cost of the opportunity cost of needing to define the solution faster than maybe your competitors.

What's more important?” 

Getting the answer fast? Getting to a product fast? Or do you have just something that's running in your background?” 


## When going full cloud, it’s important to understand what kinds of CPUs you need to for your pipeline.

<iframe width="560" height="315" src="https://www.youtube.com/embed/D1o36dL5tGQ?si=gEsVaUN5zSuENptI" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

“Depending on the shape and size of the data and the kind of scientific algorithm that you're running, there are going to be different computational requirements.

In some cases you might want a beefy VM with a lot of compute, a lot of memory. Sometimes you just, it's going to be a small operation that's just cheap, but it's more disk bound. 

So there's some very different shapes and sizes of machine that might be appropriate.” 


## Case Study: How the Broad Institute leverages spot instances to reduce costs

<iframe width="560" height="315" src="https://www.youtube.com/embed/-OEZnDIB0fs?si=BJTFvermcyj8DUcf" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

The Broad Institute saved ~90% on costs going from their first implementation to their final pipeline. By leveraging tools like Spot instances, they were able to take their costs from $85 per whole genome to $5. 

“When Broad's original genomic pipeline was migrated from on prem to cloud, the initial, completely naive implementation was about, I want to say $85 per whole genome. With some basic optimization, they very quickly brought that down to $45 per genome, which is almost half. 

Three to six months of further investment, (today it would take much less time because we have all these tools), we took it down to $5 a genome.

And if you think about the cost reduction there, that was huge. Nowadays you can use those kinds of approaches. You don't have to do six months of R&D because a lot of the tools exist. But you can really make a really big difference from the naive implementation to an optimized implementation.”



## Ready to Learn More? 
Spot instances are a great way to use spare capacity in the cloud at 60 to 80% cheaper. 

If you're curious about seamlessly running your research and other types of computational workloads on spot instances, feel free to [book a chat with me](https://calendly.com/jing-memverge/30-minute-general-discussion?month=2023-12&date=2023-12-05). 


## More about Jing
I'm Jing, I run GTM for our memory snapshot product at MemVerge.

<iframe width="560" height="315" src="https://www.youtube.com/embed/gW2uf5-LEYM?si=-ND4_wi-YSGYxGAb" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

We protect your workloads from being taken away or reclaimed by the cloud provider, even when they're not done running. And we are automatically finding you additional instances where you can continue to run your work to completion.

I love helping teams who may not have the resources to conduct their computationally intensive research locally or may not feel fully comfortable migrating their omics workflows to the cloud. 

I post regularly about running omics workflows on spot instances and in the cloud. Check out my writings on [LinkedIn](https://www.linkedin.com/in/jingxie/).

## Abstract

This blog provides the evolution and the overview of the Memverge JuiceFlow solution, which is meant for highly performant and cost-effective Nextflow pipeline executions on Memory Machine Cloud. 

---

## What is JuiceFlow?

JuiceFlow is a transient, head-node setup that utilizes **JuiceFS** as its primary filesystem to execute Nextflow pipelines on MMCloud. [JuiceFS](https://juicefs.com/docs/community/introduction/), an open-source, high-performance distributed file system designed for cloud environments, brings several unique features to the table, including:

- **Separation of Data and Metadata**: JuiceFS stores file data in chunks within object storage platforms like Amazon S3, while metadata is stored in various databases, such as Redis. This architecture enhances performance and scalability.
- **Performance**: JuiceFS is engineered to achieve millisecond-level latency and nearly unlimited throughput, depending on the object storage's scale. This makes it exceptionally suitable for high-performance computing environments like those needed for Nextflow pipelines.
- **Easy Integration with MMCloud**: MMCloud facilitates the deployment process by providing pre-configured Nextflow head node templates with JuiceFS setup, simplifying the user's experience.
- **Comparison with S3FS**: JuiceFS offers significant performance and scalability advantages over S3FS. For those interested in a detailed comparison, refer to the [JuiceFS vs. S3FS](https://juicefs.com/docs/community/comparison/juicefs_vs_s3fs/) documentation.

Once a Nextflow run concludes, whether due to failure or completion, the node will self-terminate, thus the term "transient." Furthermore, JuiceFlow can reload previous runs from your S3 bucket, provided that one metadata file exists per bucket. This feature enables you to `-resume` runs or access data created by other runs within the same JuiceFS file system, streamlining data management and pipeline execution.

---

## Development of JuiceFlow

* The development of the JuiceFS + Nextflow solution has undergone many iterations, leading to the creation of JuiceFlow.

* Initially, the setup integrated the JuiceFS filesystem and Nextflow within the same instance. However, upon deployment via CLI, users were required to `ssh` into the instance to initiate their run and also manually dispose of the VM after execution.

* In the second iteration, we emphasized the "transient-ness" of the Nextflow head-node while aiming for minimal user intervention. The JuiceFS and Nextflow components were allocated to two separate instances. In this setup, JuiceFS remained persistent, whereas the Nextflow component mounted the filesystem and stayed active until the Nextflow run concluded, underscoring its "transient" aspect. This arrangement eliminated the need for users to `ssh` into the head-node instance, allowing them to interact primarily through two CLI command submissions.

* The ultimate configuration, **JuiceFlow**, merges the JuiceFS and Nextflow components into a single node, akin to the original setup, yet it inherits the "transient" hands-off characteristic of the second iteration. This was achieved through a singular CLI command.

---

## Overall JuiceFlow Setup

* The prerequisites for setting up include a functional MMC OpCenter, an AWS S3 Bucket, and the corresponding S3 access and secret keys.

* The user is required to submit a single command that encapsulates the aforementioned details along with two primary scripts: 
 - `transient_JFS.sh`: Formats the work directory S3 bucket to JuiceFS format.
 - `job-submit.sh`: Contains Nextflow input parameters and MMC config.

* Upon submission, a VM will be initiated. This VM is responsible for formatting a directory within the designated S3 bucket for JuiceFS. If the bucket contains a metadata file from a previous job, it will load the corresponding JuiceFS partition, granting the job access to that specific directory. Within the job, a Nextflow configuration file specific for MMCloud using the `nf-float` plugin is generated, and the designated Nextflow command is executed.

* The process integrates the use of the [nf-float](https://github.com/MemVerge/nf-float) plugin, which facilitates the spawning of additional instances for each process within the pipeline on MMCloud. These VMs are configured to terminate upon the completion of their respective processes.

* Following the execution of the pipeline, regardless of the outcome (failure or success), JuiceFlow will archive the metadata of the JuiceFS filesystem to the user's work directory S3 bucket. Should JuiceFlow be executed again using the same bucket, it will recognize the metadata file and reload the identical JuiceFS Filesystem. This feature is particularly beneficial for instances where the `-resume` command is utilized in Nextflow executions.

* For a more detailed setup, please refer to the [JuiceFlow AWS QuickStart Guide](https://www.mmcloud.io/resources/docs/juiceflow).

---

## Important Considerations

- **JuiceFS Bucket Requirements**: JuiceFS cannot be formatted on a sub-directory of a bucket. The root directory of the bucket, or the bucket itself, must be specified in the CLI command.

- **Google Cloud Platform Access Requirements**: For GCP users, the service account must have permissions to get and put objects into your `gs://` bucket.

- **Handling `gcp_key.json`**: This key serves as the credential for a service account with access to your `gs://` bucket. To incorporate the contents into the job, it is read into an environment variable that the container can access.

---

## Monitoring Execution Logs

### Workflow Execution Log

Once the head node job starts executing, you can monitor the Nextflow standard output in OpCenter GUI as follows:
- Click on the head-node job-ID.
- Navigate to the `Attachments` tab.
- Click "view" (eye-icon) on the `stdout.autosave` file.

![Workflow Execution Log](https://hackmd.io/_uploads/SJFcJlIlC.png)

### Individual Job Logs

To monitor workflow execution and get a detailed view of each process in the Nextflow workflow:
- Click on the Workflows dashboard in the OpCenter.

![Individual Job Logs - Workflow Dashboard](https://hackmd.io/_uploads/SyumxlIx0.png)

- Click on the workflow name. This allows you to monitor the jobs running in this workflow in a consolidated view.

![Consolidated Workflow View](https://hackmd.io/_uploads/rkpHeg8gR.png)

- To view the execution log of any particular job:
 - Click on the job-ID.
 - Navigate to the `Attachments` tab.
 - Click "view" (eye-icon) on the `stdout.autosave` file.

![View Execution Log](https://hackmd.io/_uploads/SkvG-lIx0.png)

---

<details>
<summary>Expand here for more details on work directory inspection instructions</summary>

To inspect the files in the work directory, follow these steps:

- Submit a job similar to the commands above with a script designed to "sleep forever," as shown in the script below:

```bash
#!/bin/bash
while true
do
 sleep 1s
done
```

- You can now `ssh` into the instance and inspect all files in the work directory by navigating to `/mnt/jfs`. There's no need to enter the container since JFS is mounted to the instance itself. It's also possible to create a template for easier execution, but ensure to specify the correct bucket in the ENV variables.

**NOTE**: This method assumes that no files in the JFS mount are being changed, as we are not updating the metadata in this setup. This is intended only for reading/copying logs. Should you alter anything in the file system, be aware that changes will NOT be saved unless the metadata is updated.

</details>

---

## Performance Metrics

* Below are some key performance & cost metrics of JuiceFlow using the `test_full` profiles of popular nf-core pipelines

| Pipeline | Execution Time | Execution Cost ($) |
| --------------- | -------------- | ------------------ |
| nf-core/sarek | 9hr 32min | 6.3288 |
| nf-core/atacseq | 4hr 22min | 6.6143 |
| nf-core/rnaseq | 8hr 00min | 21.8443 |

NOTE: All pipeline jobs were run on AWS Spot instances, utilizing MMCloud's spot-protection feature. This feature automatically saves job progress upon spot instance preemption and restores it on a new spot instance, ensuring continuity.

* We previously published detailed benchmark results on performance of JuiceFS for Nextflow Workflows in a blogpost that can be viewed [here](https://www.mmcloud.io/blog/unveiling-the-superiority-of-mmcloud-over-aws-batc)

---

## JuiceFlow Advantages

- **Transient node**: Once the Nextflow pipeline has finished running, the entire node is terminated, eliminating further costs for the user.

- **One-command submission**: JuiceFlow allows the pipeline to be executed with a single command, negating the need for the user to `ssh` into the instance and thus providing a hands-off experience.

---

## Summary

JuiceFlow revolutionizes Nextflow pipeline executions on MMCloud by leveraging JuiceFS for high-performance, cost-effective computing. It introduces a seamless integration that drastically reduces execution times and costs, exemplified by nf-core pipelines running on AWS Spot instances with impressive savings and efficiency. The system's transient nature ensures that nodes terminate post-execution, eliminating unnecessary expenses. Furthermore, JuiceFlow's one-command setup significantly simplifies user interaction, reinforcing its position as a superior choice for executing complex computational workflows with optimal resource utilization and budget control.

### What is application observability and why it is important

Application observability gains insights into the behavior of software applications utilizing the information from logs, metrics and traces. The goals include but not limited to

- identify bottlenecks and optimize performance
- identify issues and troubleshoot problems quickly
- identify resource requirements and determine the most appropriate corresponding infrastructure

### What is Memory Machine Cloud WaveWatcher and how it can help

The Memory Machine Cloud WaveWatcher service builds on its ability to track CPU, memory, network, storage (size, throughput, and IO) used by a ‘job’ (could be an application, a workflow, or a pipeline). Take the below WaveWatcher result of a Sentieon WGS pipeline as an example:

![WaveWatcher for Sentieon WGS Pipeline](https://baywatch-api.memverge.com/media/upload/202311/231120214738149768.png)

We can see how this pipeline can be characterized into different steps, each with corresponding ‘behaviors.’ The resources the steps consume are shown in the table below:
| Step | Function | Observability - Memory Usage | Observability - CPU Usage | Observability - Network | Observability - Storage Usage | Observability - Storage IO | What can / should we do? |
| - | - | - | - | - | - | - | - |
1 | Validate license and download reference and sample data | extremely low | extremely low | extensive | increasing | extensive | Use a less memory and CPU intensive but network and storage optimized instance |
2 | BWA-MEM | continuously growing | extremely high | none | increasing | medium | Use a memory and CPU optimized instance and larger instance
3 | Dedup | dynamically changing | dynamically changing | zero | dynamically changing | dynamically changing | Use a CPU optimized instance and larger instance |
4 | Variant calling, filtering and genotyping | high | extremely high | zero | retain | zero | Use a memory and CPU optimized instance and medium instance

### Go beyond observability - Memory Machine Cloud WaveRider

It is helpful to gain observability but the ultimate question is - what can be improved, and can it be automated? The data from WaveWatcher can then be used by the WaveRider service to automatically right-size resources, keep the application / pipeline / workflow state, and reduce CO2 footprint, hence to

- avoid under-provision which may result application performance bottleneck or even OOM, and
- avoid over-provision which spends unnecessary money and CO2 footprint.

More detailed Sentieon WGS pipeline benchmark can be found [here](https://www.mmcloud.io/blog/sentieon-genomics-tools-boosting-performance-wit-3).

### Three pillars of Memory Machine Cloud - observability (WaveWatcher), right-sizing (WaveRider), and job automation (Float)

Last but not the least, how to automate the whole process with minimum human intervention? Just like the three pillars of observability (logs, metrics and traces), there are three pillars for Memory Machine Cloud - job automation (Float), observability (WaveWatcher), and right-sizing (WaveRider).

- The user submits a job (for example, a bioinformatics analysis pipeline) via Float CLI or GUI.
- Memory Machine Cloud WaveWatcher automatically observes the behavior including the resources needed
- Memory Machine Cloud WaveRider automatically right-sizes / vertical scaling based on WaveWatcher’s observation supporting stateful job

These three pillars make a complete cloud automation solution removing the hassle of integrations among multiple solutions.

### Comparison of Memory Machine Cloud with modern solutions

Below is a summary table of comparison of Memory Machine Cloud with modern solutions, where Memory Machine Cloud provides an all-in-one solution.
| Category | Comparison | MemVerge MMCloud | Grafana | Datadog | Spot by NetApp | YellowDog | AWS CloudWatch | AWS Compute Optimizer | AWS Batch |
|:----------:|:---------------------------------------------------------------:|:------------------:|:----------:|:----------:|:--------------:|:---------------:|:--------------:|:---------------------:|:----------:|
| Feature | Observability | Yes (WaveWatcher) | Yes | Yes | Yes | Yes | Yes | Yes | No |
| Feature | Automatic right-sizing stateless application | Yes (WaveRider) | No | No | Yes | Yes | No | No | No |
| Feature | Automatic right-sizing stateful, non-fault-tolerant application | Yes (WaveRider) | No | No | No | No | No | No | No |
| Feature | Job automation | Yes (Float) | No | No | No | Yes | No | No | Yes |
| Deployment | On-prem version | Yes | Yes | No | No | Yes | No | No | No |
| Pricing | Pricing | All-in-one license | By license | By service | By license | Basic + add-ons | By service | Opt-in | By service |

\*note - more detailed comparisons with Spot by NetApp can be found [here](https://www.mmcloud.io/blog/memory-machine-cloud-vs-spot-by-netapp)

### Call for action - save your spending and time

Do you need to run stateful, non-fault-tolerant workloads or pipelines on the cloud? Would you like to benefit from automatic right-sizing and vertical scaling and reduce both cloud cost and job running hours without worrying about integrations among multiple solutions?  [Please try Memory Machine Cloud for free](https://www.mmcloud.io/customer/register).

High-performance computing (HPC) traditionally involves clusters of compute servers deployed in dedicated facilities. In recent years, demands for scale and performance have pushed HPC from on-premises to the cloud - a transition that offers new opportunities as well as significant hurdles.

On the positive side, the cloud offers virtually unlimited on-demand computing and storage resources because cloud platforms provide elastic capacity that can grow (and shrink) dynamically when resource demands change.

There are economic benefits to moving to the cloud, although not always. A stable workload that runs 24x7x365 for several years may be cheaper to run on dedicated hardware (cloud service providers have responded to this case by offering bare-metal instances). The pay-as-you-go model eliminates upfront capital expenses and avoids the costs associated with unused compute cycles. The range of compute instance types available in the cloud allows users to pay only for what they need, when they need it.

However, running applications in public clouds introduces new challenges. Each cloud service provider has an extensive catalog of services, and each service has a wide array of features. Without the knowledge required to configure and optimize cloud services, the researcher may encounter unpredictable or unsatisfactory application performance, unexpected costs, or unnecessary security risks.

This is where Memverge’s Memory Machine Cloud (MMCloud) can help. MMCloud is an easy-to-use cloud automation platform designed for scientific computing and data science. MMCloud simplifies the deployment of data-intensive pipelines and interactive computing applications in the cloud through automation and containerization.

Containers are the key technology that enables application mobility and automates the deployment of applications, at scale, in the cloud. The container includes all the software dependencies and runtime libraries needed to run the application in a reproducible way. But container deployment requires a cloud orchestration platform to schedule, instantiate, and manage containers. MMCloud provides an intuitive GUI (as well as a CLI) that allows the non-cloud specialist to deploy complex workflows as containerized applications in the cloud.

There are many reasons why scientists rely on MMCloud to enhance their research capabilities. Here are the top five reasons why you should choose MMCloud as the platform to run your cloud workloads.

**1. Seamless Expansion of High-performance Computing to the Cloud**

Hybrid cloud (a combination of on-premises and cloud resources) is a popular architecture for HPC because of its flexibility and scalability. When demand for computing resources exceeds what is available locally (on-premises), applications can “burst” to the cloud.

MMCloud effectively augments the local HPC environment with public cloud by enabling the user to launch jobs in the cloud using the same familiar HPC job managers such as SLURM and PBS. This way, scientists can operate in an efficient hybrid environment. Containerization and automation are a core feature of MMCloud. MMCloud automates deployment of containerized applications into the clouds, and it automatically deletes cloud resources after a job completes, removing the risk of unattended resources adding to the monthly cloud bill. Interactive sessions can be hibernated and restarted when needed again.

![Memory Machine Cloud HPC Bridge extends HPC into the cloud](https://baywatch-api.memverge.com/media/upload/202312/231207200802821427.jpeg)


Figure 1 MMCloud expands HPC to clouds

**2. Insightful Monitoring of Cloud Workloads**

Some users are reluctant to move workloads to the cloud because of a concern that they will have limited insight into, and control over, how their applications run in the cloud. Will the job complete in a shorter or longer time? Are the resources over- or under-provisioned?

Visibility into workload performance and resource utilization is critical when operating in the cloud. Granular monitoring of resource usage and costs can identify opportunities for optimization through rightsizing, shutting down idle resources or shifting workloads based on the current prices of compute instances. While cloud service providers do offer monitoring tools, these tools are not always easy to use and often come with additional costs.

MMCloud provides a comprehensive solution with advanced monitoring and metering capabilities, ensuring that scientists have a detailed understanding how their workloads run. CostReporter gives you a summary how much you spend on the jobs running in the cloud. WaveWatcher makes it easy to see the real-time consumption of resources including CPU, memory, storage, and network transfer speeds. For example, an application may exceed its storage limits or crash because of insufficient resources. Issues like these can be identified using MMCloud’s graphical interface, making troubleshooting much more straightforward.

MMCloud generates event and performance logs at both the job and host (worker node) level. Logs can be viewed in real-time to track the progress of a job as it executes, or downloaded after a job completes to identify issues or errors. The monitoring features enable researchers to fine-tune their environments for maximum efficiency.

![MMCloud WaveWatcher provides insights to cloud workloads](https://baywatch-api.memverge.com/media/upload/202311/231121040115584000.png)

Figure 2 MMCloud WaveWatcher provides insights to cloud workloads

**3. Reliable Spot Instances**

Almost all service providers offer spot Instances as a way for users to purchase excess computing capacity at a significantly lower price than on-demand instances (discounts can be as much as 90%). The trade-off is unpredictable availability. The cloud service provider can reclaim these instances with nominal warning (usually two minutes or less). Spot instances are ideal for stateless applications and can be used in stateful, batch-oriented pipelines if the workflow manager can requeue any failed jobs, although the total cost and wall clock time become unpredictable.

MMCloud's SpotSurfer feature allows stateful applications to always run to completion by "surfing" to a new VM instance if the underlying Spot Instance is reclaimed. Customers can realize the cost savings of spot instances without the risk of job interruptions. The technology that makes SpotSurfer possible is AppCapsule. AppCapsule creates a snapshot of the in-memory state as well as file system data. AppCapsule is triggered automatically when the cloud service provider signals that it is reclaiming the spot instance. Job execution pauses and then resumes on a new spot instance.

For interactive sessions, MMCloud provides a reverse proxy to ensure that each interactive session appears to clients as an IP address that does not change even if the session moves to a different virtual machine with a different IP address.
![MMCloud SpotSurfer protects workloads on spot instances](https://baywatch-api.memverge.com/media/upload/202311/231121040319580845.png)

Figure 3 MMCloud SpotSurfer protects workloads on spot instances

**4. Cost Control**

Cost management is a major concern for researchers, many of whom are grant-funded with strict budget constraints. While cloud services can be cost-effective, they can also lead to unpredictable expenses if not managed carefully. Institutions must develop new financial governance models to control cloud spending, ensuring that costs align with research budgets. Managing costs at cloud scale demands continuous monitoring of cloud resource usage to guard against runaway expenses. MMCloud’s Job Cost Reporting tools allow for precise budgeting, and the SurfZone feature provides real-time enforcement of limits on cloud spending.

Lifecycle management of cloud-based workloads can be critical for cost control. MMCloud provides control over of resources, for example, by enforcing automated schedules for job startup and shutdown, and maximum job execution times. This level of control is essential for managing costs in group settings where multiple users may run concurrent experiments. There are actions that individual users can take, for example, canceling jobs, moving jobs to smaller (cheaper) instances, or suspending interactive sessions. These actions can result in saving cost and can be used for avoiding idle charges.

Performance can be critical for cost control too. When jobs run on the spot instances, the slower jobs in data pipelines have a higher chance of getting reclaimed and the time and expense penalty for starting over might be huge. MMCloud’s integration with third-party high-performance file systems has improved the efficiency of file operation and has shown a significant advantage for Nextflow workflows.

WaveRadar, an interactive dashboard created on mmcloud.io is a free FinOps tool for viewing historical spot EC2 prices across all major AWS regions. WaveRadar lets you identify the regions offering the lowest rates for a given instance or instance family.

**5. Rightsizing Cloud Resources**

Compute instances available from cloud service providers range in size from one vCPU and 500 MB memory to 192 vCPUs and 1.5 TB memory (and even more in some specialized instances). Choosing the optimal cloud resources for scientific workloads is a complex task. Sizing for the peak load is safe but inefficient. Sizing for the average load reduces the cost but can lead to unacceptable performance or even out-of-memory (OOM) errors.

MMCloud's WaveRider feature simplifies this process by removing the need to select a single instance for each task. WaveRider provides the capability to migrate running jobs to optimally sized virtual machines based on their real-time resource utilization.

Job migration can be initiated in three ways: manually, driven by policy, or programmatically. At any time, you can manually migrate a job to a specific instance type, or to an instance whose capacities you define. With a rules-based policy, WaveRider monitors the resource utilization of your workload. If the upper threshold for CPU or memory utilization is crossed for a specified interval, the job is migrated to a virtual machine that has more virtual CPUs or more memory. Similarly, if the lower threshold is crossed and utilization remains low for a specified interval, the job is moved to a smaller virtual machine. For jobs with a well-known resource usage profile, job migration can be programmed into the job script so that the job migrates at predetermined stages in the execution.

Rightsizing has a positive effect on the optimal allocation and sharing of cloud resources. The optimization of using cloud resources has a profound impact on environmental sustainability. Users who proactively release underutilized resources allow the cloud service provider to increase server utilization. Fewer servers running at higher utilization lowers operational as well as embedded emissions. In a world increasingly conscious of climate change, optimizing cloud resource usage is a step towards reducing the overall carbon footprint of the IT industry. MMCloud provides a carbon emissions dashboard so that users become aware of the impact of how they run their jobs in the cloud.

![MMCloud Carbon Emission Monitor](https://baywatch-api.memverge.com/media/upload/202311/231121040303736482.png)

Figure 4 MMCloud Carbon Emission Monitor

**Conclusion**

Moving HPC workloads to the cloud, or bursting workloads to the cloud when needed, has benefits in terms of cost, scalability, efficiency, and sustainability. Challenges in usability and cost control in the cloud have prevented users from taking advantage of the cloud or a hybrid cloud architecture.

MMCloud is a powerful, yet easy to use, platform that facilitates the use of cloud computing by the research community. MMCloud is cloud-agnostic and is supported on AWS, Google Cloud, and AliCloud today. A free 30-day, no obligation trial is available. Register at [https://mmcloud.io/customer/register](https://mmcloud.io/customer/register) today.

In the previous <a href="https://www.mmcloud.io/blog/surfzone-using-memory-machine-cloud-to-prevent-clo" rel="noopener noreferrer" target="_blank" style="color: var(--ds-link, #0052CC); background-color: rgb(255, 255, 255);">SurfZone blog</a>, we talked about some real-world examples of cloud budget overruns and their painful consequences. We also mentioned the limitations of existing solutions and how SurfZone can overcome them by providing timely enforcement while saving interim progress.In this blog, we describe the user journeys of both the Cloud Admin and the Cloud User. <h3>Typical user journey</h3>A typical high-level user journey takes 3 steps -<ol><li data-list="ordered">Cloud Admin defines quota group</li><li data-list="ordered">Cloud Admin applies quota group to Cloud User and Cloud Users' submitted jobs will automatically be enforced by the policy of the given quota group.</li><li data-list="ordered">Cloud Admin and Cloud User get notifications when Cloud User's consumption reaches the quota limit or any quota-related actions occur.</li></ol><img src="https://baywatch-api.memverge.com/media/upload/202310/231023012347452899.jpeg"> Now let’s take a closer look at each of the steps along with Memory MachineTM Cloud OpCenter Web GUI.Step 1 - Cloud Admin or Principal Investigator defines a quota group (see Figure 1.1) as follows, and views list of created quota groups (see Figure 1.2).<ol><li data-list="ordered" class="ql-indent-1">What is the quota type you would like to monitor (for example, cost)? What is the quota limit?</li><li data-list="ordered" class="ql-indent-1">What is the usage threshold for notification?</li><li data-list="ordered" class="ql-indent-1">What action should be taken when the usage exceeds the quota? Would it be</li><li data-list="ordered" class="ql-indent-2">Cancel the job, or</li><li data-list="ordered" class="ql-indent-2">Suspend the job (where you can enable auto resume)</li><li data-list="ordered" class="ql-indent-1">What action should be taken when additional quota is applied (for example, auto-resume)?</li></ol><img src="https://baywatch-api.memverge.com/media/upload/202310/231019182008359212.png">Figure 1.1. Cloud Admin adds/edits SurfZone&nbsp;<img src="https://baywatch-api.memverge.com/media/upload/202310/231019182028234291.png">Figure 1.2. Cloud Admin views the list of SurfZone&nbsp;Step 2 - Cloud Admin applies the quota group to a Cloud User (see Figure 2.1 and Figure 2.2) - the Cloud User's jobs are automatically subject to the enforcement mechanisms defined by the quota group (see Figure 2.3).<img src="https://baywatch-api.memverge.com/media/upload/202310/231019182141403006.png">Figure 2.1. Cloud Admin applies SurfZone to Cloud User&nbsp;<img src="https://baywatch-api.memverge.com/media/upload/202310/231019182152185529.png">Figure 2.2. Cloud Admin views the list of Cloud User including quota and usage&nbsp;<img src="https://baywatch-api.memverge.com/media/upload/202310/231019182400294309.png">Figure 2.3. Cloud Admin views the list of Job including their status&nbsp;Step 3 - Cloud Admin (or Principal Investigator) and Cloud User get notifications when user's consumption reaches the quota limit or any quota-related actions occur.<img src="https://baywatch-api.memverge.com/media/upload/202310/231019182429989309.png">Figure 3.1. Cloud Admin gets notifications on users whose quota usage above the configured threshold <img src="https://baywatch-api.memverge.com/media/upload/202310/231019182439198049.png">Figure 3.2. Cloud User views his/her own quota and usage&nbsp;<h3> </h3><h3>What’s next?</h3>What are your thoughts on this feature?If you would like to be on the waitlist to preview this feature, leave your contact information in the comments below.<a href="https://www.mmcloud.io/" rel="noopener noreferrer" target="_blank" style="color: var(--ds-link, #0052CC); background-color: rgb(255, 255, 255);">Memory Machine Cloud by MemVerge</a>

<h2>Variability in AWS EC2 Spot Instance Discounts</h2><h2>Executive Summary</h2>AWS customers can reduce cloud costs by using <a href="https://www.mmcloud.io" rel="noopener noreferrer" target="_blank">Memory Machine Cloud</a> to deploy their containerized applications on spot instances (EC2 instances offered at a discount to on-demand instances). <a href="https://www.mmcloud.io/blog/waveradar-see-and-understand-ec2-spot-price-histor" rel="noopener noreferrer" target="_blank">WaveRadar</a> is a useful (free) online tool for examining historical spot instance prices and discounts. For a given EC2 instance type, the discount depends on several parameters. The analysis reported here, using data from WaveRadar filtered for spot instances available in the US, shows that discounts vary strongly with region, instance family, and CPU vendor. Other factors, such as instance size and CPU generation, also impact discounts but the effects are less.Generally, discounts are smallest in us-east-1 and us-west-2, and greatest in us-east-2 and us-west-1. However, not all EC2 types are available as spot instances in us-west-1. AMD-powered instances generally have smaller discounts than Intel-powered instances although the Intel discounts are more variable. If maximizing discount is the only selection criterion, then deploying Intel-based instances in us-east-2 is recommended. In practice, there are additional factors that affect cost and performance, such as networking and data transfer.Discounts for memory-optimized instances are larger than discounts for compute-optimized instances with discounts for general-purpose instances in between.The current inventory of spot instances, and their reclaim probabilities, are additional factors that may influence spot instance selection. Detailed data on inventory and reclaim probability is not available, so those aspects are not discussed here.<h2>AWS Compute Instances</h2>Elastic Compute Cloud (EC2) is the cloud service that provides virtual machines to AWS customers. EC2 instances are available in different types where each type is identified by a name (for example, c6i.4xlarge) and each type represents a configuration of physical CPU (for example, Intel Xeon 8375C), virtual CPUs (for example, 16 vCPUs), and memory (for example, 32 GiB).EC2 instance types are grouped into families, including:<ol><li data-list="bullet">General purpose</li><li data-list="bullet">Compute optimized</li><li data-list="bullet">Memory optimized</li></ol> Most EC2 instances follow the same naming convention.<code>[family][generation][CPU-vendor]</code>.<code>[size]</code>largewhere<ol><li data-list="bullet"><code>[family]</code> is a letter to indicate general purpose, compute-optimized, or memory-optimized</li><li data-list="bullet"><code>[generation]</code> is an integer</li><li data-list="bullet"><code>[CPU-vendor]</code> is a letter to indicate Intel, AMD, or Graviton</li><li data-list="bullet"><code>[size]</code> is a string that identifies a combination of number of vCPUs and memory</li></ol> For example, r6i.4xlarge is a memory-optimized, 6th generation, Intel-powered, instance with 16 vCPUs and 128 GiB of memory.EC2 instances are distributed geographically among the regions where AWS locates its data centers. Within a region, EC2 instances are distributed among availability zones, which are logically separate networks that provide fault isolation. Currently, AWS operates 102 availability zones within 32 geographic regions around the world.<h2>Price Lists</h2>EC2 instances are billed in one-second increments, although the prices are displayed on the AWS website as hourly rates. There are three price lists that apply to EC2 instances.<ol><li data-list="bullet">On-demand prices apply to instances that are configured without prior reservations, can run for any length of time, and carry no long term commitments.</li><li data-list="bullet">Reserve prices apply to instances that are associated with commitments to pay for a minimum number of EC2 instances for a fixed period (1 or 3 years) with constraints on the type and location of each instance.</li><li data-list="bullet">Spot prices apply to EC2 instances that are only available because AWS has excess capacity at that time. Spot prices vary as supply and demand change.</li></ol> For a given EC2 instance type, the on-demand price is the highest and the spot price is the lowest (the reserve price is in between).<h2>EC2 Price Data</h2>AWS provides a Price List API to programmatically retrieve information on AWS products and pricing. By querying the AWS Price List API multiple times a day and inserting the results into a database, WaveRadar assembles a historical record of on-demand and spot instance prices for all EC2 instances in all regions.On-demand prices change slowly — prices changes only occur at the start of each month and prices may remain unchanged for several months. Spot prices are volatile but in practice, changes are only visible when viewed in a three month window (you can view EC2 price history from your AWS management console).EC2 instances are available with a choice of operating system: a generic, unsupported Linux version (labeled Linux/UNIX or AWS Linux), two supported Linux versions (SUSE and Red Hat), and Windows. Of the four operating systems, Linux/UNIX (AWS Linux) has the lowest price (Windows has the highest).<h2>The Ideal Search</h2>The search for the optimal EC2 instance to run a job is a complex task. A user might not know the peak resource requirements (CPU clock speed, number of vCPUs, and memory size) for a job or how to map resource requirements to an EC2 instance at the best price. In a simple case, a user may ask: for a given EC2 instance type, where can I get the lowest price for this type as a spot instance? A more complex query may search for optimal price performance, that is, the best price per vCPU, or per GiB, adjusted for processor generation.AWS offers spot instances at a discount as a way to monetize excess EC2 capacity. That means that not every instance type is available as a spot instance at any given time, and any spot instance can be reclaimed with nominal warning. The ideal search would include the current inventory of every spot instance type to allow the user to estimate the probabilities of reclaim events. Inventory information is not available via the AWS API, so discounts must be used as an indirect indicator.<h2>Simplifying Assumptions</h2>The studies described in this report focus on finding the largest discounts on spot instances configured with Linux/UNIX (AWS Linux) in the regions that AWS operates in the continental United States.<h2>Variability in Discounts</h2>All functions in <a href="https://www.mmcloud.io" rel="noopener noreferrer" target="_blank">Memory Machine Cloud</a> (MMCloud) are controlled by the Operations Center (OpCenter). One OpCenter can deploy many jobs in a region but one OpCenter cannot span regions. Although an MMCloud customer could deploy OpCenters in multiple regions, it is more common to deploy one OpCenter and run all jobs in a single region. An OpCenter is deployed in one availability zone but jobs are deployed in all availability zones in the region.Containers deployed by the OpCenter run on EC2 instances supported by Intel or AMD processors. As Intel and AMD develop new generations of processors, AWS makes new generations of EC2 families available. For example, the C5 family of EC2 instances runs on 2nd generation Intel Xeon Scalable Processors (Cascade Lake) or 1st generation Intel Xeon Platinum 8000 series (Skylake) processors. The C6i family runs on 3rd generation Intel Xeon Scalable processors and the C7i family runs on 4th Generation Intel Xeon Scalable processors. Later generations of processor offer better price performance than earlier generations. A similar pattern applies to AMD-based families, for example, C5a and C6a, or R5a, R6a, and R7a.With this background, we identify potential sources of variability in spot prices (and hence, a path towards finding the largest discounts), namely,<ol><li data-list="bullet">Region (us-east-1, us-east-2, us-west-1, or us-west-2)</li><li data-list="bullet">EC2 instance family (M, C or R)</li><li data-list="bullet">EC2 instance family generation (5, 6 or 7)</li><li data-list="bullet">CPU vendor (AMD or Intel)</li></ol> To preview the data, we survey the discounts for each EC2 instance family aggregated across all regions (see Figure 1). Figure 1 shows that the majority of the discounts fall between 20% and 75% (multiply by 100 to convert the discount to a percentage), with C instances towards the lower end, R instances towards the upper end, and M instances in the middle.<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211145336885.png">Figure 1. Discounts for M, C, and R instance families<h2>Discount Variation by Region</h2>To investigate how discount varies by region, we hold the instance family (M, C, or R), generation (5), and CPU vendor (a denotes AMD) constant while querying discounts in each region. The results for the M5a family, the C5a family, and R5a family are in shown in Figures 2 to 4, respectively. Multiply the discount by 100 to convert to a percentage.<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211241021492.png">Figure 2. Discounts for M5a family of instances by region<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211309865110.png">Figure 3. Discounts for C5a family of instances by region<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211340069740.png">Figure 4. Discounts for R5a family of instances by regionThe results show that, although there are outliers, discounts in us-east-1 and us-west-2 are similar whereas discounts in us-east-2 and us-west-1 are similar. The same pattern is seen for Intel-powered instances.For the remaining queries, we treat us-east-1 and us-west-2 as a single group, and us-east-2 and us-west-1 as a single group.<h2>Discount Variation by EC2 size</h2>To investigate how discount varies with instance size, we hold the instance family (M, C, or R), generation (5, 6, or 7), and CPU vendor (a denotes AMD and i denotes Intel) constant while querying discounts for each <code>[size]</code>. The results for the M5a family, the C5a family, and R5a family are in shown in Figures 5 to 7, respectively.<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211409811895.png">Figure 5. Discounts for M5a family of instances by size<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211430566036.png">Figure 6. Discounts for C5a family of instances by size<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211457040980.png">Figure 7. Discounts for R5a family of instances by sizeThe results show that, although there are outliers in the C5a family, discounts do not vary greatly from one instance size to another in the same family. Discounts do vary between families: the R5a family has the greatest discounts, the C5a family has the lowest discounts, and the M5a family is in the middle.For the remaining queries, we treat large, xlarge, 2xlarge...48xlarge instances within a family as a single group.<h2>Discount Variation by CPU vendor</h2>For most of the EC2 families, there is an Intel-powered version and an AMD-powered version of every instance. To investigate whether discounts vary by CPU vendor, we repeat the queries behind Figures 5, 6, and 7 for the Intel-powered versions. Compared to the equivalent AMD-powered instances, we see that discounts for Intel-powered instances are larger and more variable. The detailed results are shown in the Appendix ( see Figures A1, A2, and A3).If all the generation 5 family types are aggregated into separate Intel- and AMD-powered groups, the results show that on average Intel-powered instances are offered at higher discounts than AMD-powered instances (see Figure 8).<img src="https://baywatch-api.memverge.com/media/upload/202310/231010211935867992.png">Figure 8. Discounts for generation "5" instances by CPU vendor<h2>Discount Variation by Generation</h2>To investigate how discounts change as newer processors become available, we compare CPU generations while keeping the group of families constant, for example, we group {M5a, C5a, R5a} into one group and compare that group's average discount with the {M6a, C6a, R6a} group's average discount and the {M7a, C7a, R7a} group's average discount. The results for AMD-powered instances are shown in Figure 9 and the results for Intel-powered instances are shown in Figure 10.<img src="https://baywatch-api.memverge.com/media/upload/202310/231010212037208725.png">Figure 9. Discounts for groups of families by CPU generation (AMD)<img src="https://baywatch-api.memverge.com/media/upload/202310/231010212115384416.png">Figure 10. Discounts for groups of families by CPU generation (Intel)The results for the AMD-powered group show that the latest generation (7) has the largest discounts. As expected, the Intel results are more variable. Although the average discount for the latest generation (7) is lower, there are outliers that offer larger discounts than either generation 5 or 6.<h2>Conclusion</h2>AWS customers can reduce cloud costs by using <a href="https://www.mmcloud.io" rel="noopener noreferrer" target="_blank">Memory Machine Cloud</a> to deploy their containerized applications on spot instances, which are offered at a discount to on-demand instances. For a given EC2 instance type, the discount depends on a number of parameters. The analysis reported here shows that discounts vary strongly with region, instance family, and CPU vendor. Other factors, such as instance size and CPU generation, also impact discounts but the effect is less.Future analysis will investigate other factors (such as availability zone and day of week) and interactions between factors, for example, instance size and CPU generation.<a href="https://www.mmcloud.io/blog/waveradar-see-and-understand-ec2-spot-price-histor" rel="noopener noreferrer" target="_blank">WaveRadar</a> can be used to explore the data further.<h2>Appendix</h2>To investigate how discount varies with Intel-powered instance size, we hold the instance family (M, C, or R) and generation (5, 6, or 7) constant while querying discounts for each <code>[size]</code>. The results for the M5 family, the C5 family, and R5 family are in shown in Figures A1 to A3, respectively.<img src="https://baywatch-api.memverge.com/media/upload/202310/231010212152522850.png">Figure A1. Discounts for M5 (Intel) family of instances by size<img src="https://baywatch-api.memverge.com/media/upload/202310/231010212223876009.png">Figure A2. Discounts for C5 (Intel) family of instances by size<img src="https://baywatch-api.memverge.com/media/upload/202310/231010212244122999.png">Figure A3. Discounts for R5 (Intel) family of instances by size

<h3>Who should read this blog? </h3>If you are a Cloud Admin or a Principal Investigator and you are concerned about your team inadvertently exceeding their cloud budget, reading this blog is worth your time (3-minutes).Real-world examples of cloud budget overrunsHere are real-world exampl﻿es of cloud budget overruns...and the painful consequences.In a <a href="http://www.rj-robbins.com/recommended/videos/airi-2021/" rel="noopener noreferrer" target="_blank">presentation</a> on cloud computing in grant-funded research environments, Robert J. Robbins explains how the success of projects can be jeopardized by unexpected (and accidental) expenses. Here are examples.<ol><li data-list="bullet">An expense of $20,000 incurred because a researcher forgot to shut down an AWS instance after completing an analysis - the instance continued to run at $7 an hour for months.</li><li data-list="bullet">An expense of $72,000 incurred because of exponential auto-scaling caused by inserting small tests into product code - the company said they almost went bankrupt.</li></ol> Why aren’t there solutions for this serious problem?Limitations of existing solutionsThere are solutions that monitor cloud cost and can stop running instances, but there are two critical limitations.<ol><li data-list="bullet">Not timely - AWS Budgets, for example, can track cost and usage, but updates typically occur at 8–12 hour intervals. The cloud budget overrun may happen before the next update.</li><li data-list="bullet">Interim progress lost - even if you are alerted when the cloud budget is exceeded and you stop all running instances, interim progress made by unfinished jobs is lost. This means not only the money but the time has been spent and wasted.</li></ol> Is there a single solution to overcome both of these two critical limitations?SurfZone: Memory Machine Cloud's solutionMemory Machine Cloud's SurfZone overcomes both limitations.<ol><li data-list="bullet">Timely - SurfZone provides real-time enforcement for a user's cloud spend (default update interval is one hour but can be configured to a shorter or longer interval).</li><li data-list="bullet">Interim progress saved - When a user reaches the cloud budget limit, SurfZone checkpoints the current application state and puts the application into sleep mode. The application automatically resumes when additional budget is applied.</li></ol> Typical user journeyA typical user journey resembles the following.<ol><li data-list="ordered">Cloud Admin or Principal Investigator defines a quota (budget) group as follows.</li><li data-list="ordered" class="ql-indent-1">What is the quota type you would like to monitor (for example, cost)?</li><li data-list="ordered" class="ql-indent-1">What action should be taken when the usage exceeds the quota (for example, suspend or terminate the job)?</li><li data-list="ordered" class="ql-indent-1">What action should be taken when additional quota is applied (for example, auto-resume)?</li><li data-list="ordered">Cloud Admin or Principal Investigator applies the quota group to a user - the user's jobs are automatically subject to the enforcement mechanisms defined by the quota group.</li><li data-list="ordered">Cloud Admin (or Principal Investigator) and user get notifications when user's spending reaches the quota limit or any quota-related actions occur.</li></ol> What’s next?In a subsequent blog, we’ll take a deeper dive into the user interface that supports SurfZone.What do you think of this feature? Does it work for you or your organization? If you would like to preview this feature, leave your contact information in the comments below.<a href="https://www.mmcloud.io/" rel="noopener noreferrer" target="_blank" style="background-color: var(--ds-surface-raised,white); color: var(--ds-link, #0052CC);"><img src="https://www.mmcloud.io/favicon.ico">Memory Machine Cloud by MemVerge</a>

<h2>Introducing WaveRadar</h2><h3> </h3>There aren't really any great tools for understanding and comparing historical Spot EC2 prices for the 600+ instances available on AWS. To solve this, we created WaveRadar, an interactive dashboard for viewing Spot EC2 prices across all major AWS regions. WaveRadar lets you easily identify the regions offering the lowest $/hour for a given instance or instance family. It can also help you quickly compare the relative price performance $/core and $/memory of different EC2 instances at recent and historical Spot market prices. Use WaveRadar as a free FinOps tool to make more informed decisions around cost optimization, just sign up for a free account via the steps below. To access WaveRadar just navigate to the <a href="https://www.mmcloud.io/aws-spot-pricing" rel="noopener noreferrer" target="_blank" style="background-color: rgb(255, 255, 255);">dashboard_link</a>. <h3>Screenshots of WaveRadar</h3>region optimizer tab<img src="https://hackmd.io/_uploads/HysT_bWkT.png"> visualize/download data tab<img src="https://hackmd.io/_uploads/rJQxY-ZyT.png"><h3> </h3><h3> </h3><h3>Example use cases</h3>Filter/sort the tabulator in the region optimizer tab to find the right instance and region:<ol><li data-list="bullet">Finding the cheapest region for t3.medium spot instance on Sep 12 (3x difference) : <code>lowest recent spot price ($0.0125) at us-east-2b / highest recent spot price ($0.0404) at il-central-1a</code></li><li data-list="bullet">Finding the cheapest region for t3.medium on-demand instance on Sep 12 (1.5x difference) : <code>lowest on-demand price ($0.0416) at us-east-2b / highest on-demand price ($0.0672) at sa-east-1c</code></li><li data-list="bullet">Find a cheap instance with 4 CPUs and 16 GiB memory in us-west-1c on Sep 14 : <code>lowest recent spot price ($0.0480) for t4g.xlarge / highest recent spot price ($0.0662) for t2.xlarge </code></li></ol> Visualize and download spot price history in the visualize/download data tab:<ol><li data-list="bullet" class="ql-indent-1">Access historical spot price data from April to the current date for informed decision-making.</li></ol>

<h2>Cloud’s promise unfulfilled</h2><img src="https://miro.medium.com/v2/resize:fit:1400/1*jCza7dtD08vfltfQw2XhYw.png" height="207" width="700">Recent news suggest that the growth of cloud adoption is encountering serious impedance. There are even organizations repatriating compute resources back to on-premises data centers, and cloud cost seems to be the culprit.Indeed, a 2 CPU (24 core)/48 GB RAM server sells for ~$10,000. On AWS, a 24 vCPU / 48 GB RAM VM costs ~$1.18/hr. If you continuously ran the VM for 1 year, you would have paid $10,195 and could have bought the server hardware. Since the typical useful life of server hardware is 4–5 years, you would’ve overpaid 4–5 times for the server. Of course in a more rigorous comparison we need to consider operational costs but I think we all agree that continuously running the cloud server the same way as an on-prem server is costly.The key word, of course, is “continuously.” To reap the benefit of the cloud you need to use it efficiently, i.e., releasing resources that you are not using, and using resources that match the needs of your applications.<h2>Several challenges stand in the way of efficient cloud usage in the enterprise:</h2><h2>App owner access to the cloud</h2>The team that manages the cloud resources, i.e., the cloud admin team, is not the team that is running the applications, i.e., the application owners: the developers, the analysts, the scientists, et al. The application users often need to wait hours or days to gain access to cloud resources, and they don’t have the tools or the discipline to release the resources as quickly as possible.<h2>App owner flying blind</h2>The application owners may not know the most appropriate machine size to use for their applications. Often, they do not have the tools necessary to view the resource utilization of the application. They only know their cloud bill is high, their application seems to take a long time to complete, they suffer out-of-memory errors, and their application crashes and must be restarted.<h2>Forced to provision for peak usage</h2>Even when the application owner understands the application, it is often difficult to pick one machine size because most applications have peaks and troughs in their resource utilizations. If one picks the machine size that can handle the peak utilization, it often means the machine is under-utilized most of the time. If one sacrifices performance by under provisioning, then the application may crash (OOM) or otherwise run very slowly.<h2>The serverless model for complex, stateful apps</h2>When we set out to build a product for the cloud, leveraging the fruits of our research in Big Memory Computing, we asked ourselves, “Can we build software that enable all applications, including stateful apps, to be serverless?” That means application owners can focus on their subject areas and never need to worry about management of servers or other cloud resources, nor depend on another team to do so.Stateful applications should be able to take advantage of the elasticity of the cloud and consume resources most appropriate for it, even when the resource utilization is bursty. Serverless computing services, such as AWS Lambda, have proven that the serverless model is highly desirable for cloud native microservices. But how do we enable complex applications, even stateful applications, to run serverless?<h2>Decoupling complex, stateful applications from servers</h2><img src="https://miro.medium.com/v2/resize:fit:1400/0*ETvWyn5e8XFOeJGV.png" height="354" width="700">Application runtime is “chained” to serversRecent news suggest that the growth of cloud adoption is encountering serious impedance. There are even organizations repatriating compute resources back to on-premises data centers, and cloud cost seems to be the culprit.Indeed, a 2 CPU (24 core)/48 GB RAM server sells for ~$10,000. On AWS, a 24 vCPU / 48 GB RAM VM costs ~$1.18/hr. If you continuously ran the VM for 1 year, you would have paid $10,195 and could have bought the server hardware. Since the typical useful life of server hardware is 4–5 years, you would’ve overpaid 4–5 times for the server. Of course in a more rigorous comparison we need to consider operational costs but I think we all agree that continuously running the cloud server the same way as an on-prem server is costly.The key word, of course, is “continuously.” To reap the benefit of the cloud you need to use it efficiently, i.e., releasing resources that you are not using, and using resources that match the needs of your applications.<h2>AppCapsule technology makes it possible</h2>We invented a solution for decoupling stateful apps from servers with AppCapsule technology. A stateful application is one whose execution depends on the data stored in memory. Because memory is not persistent, when the server hardware reboots or changes, the data in memory is lost, and the application must start from the beginning again. Our AppCapsule creates a snapshot of all the data in memory, and all the application states required to restore the application to this moment in time, as well as the data on storage. With AppCapsule, we can transport the application runtime across different machines, even different clouds.<img src="https://miro.medium.com/v2/resize:fit:1400/0*X_gS1OCoelC4ImVA.png" height="303" width="700">AppCapsule SnapshotArmed with this technology, we set out to build control components that natively integrates with the clouds, and the user interface designed to serve the application users, and voilà, we have Memory Machine Cloud, the cloud automation platform that delivers the Serverless experience for most applications.<h2>The Floating App</h2>The internal engineering codename for Memory Machine Cloud is “Float,” because when you launch your application using Memory Machine, your application is floating on the cloud, untethered to the server it is running on. For that reason, our CLI commands all start with the keyword “float.”It’s pretty amazing what you can do when your application is floating. The first use case we tackled was running long, stateful jobs, such as genomic sequencing, on spot instances. Spot instances offer the most economical way to tap into the vast compute resources in the public cloud, but because the CSP can evict your workload on spot with a very short notice period, they are not suitable for these jobs. That is, unless you use Memory Machine Cloud.When you launch your genomic sequencing job using Memory Machine, you simply specify the spec of the machine, and that you want to leverage spot instances. Memory Machine takes care of finding the spot instances, deploying your containers, mapping your storage, and launching your job. If there is a spot eviction event, Memory Machine creates an AppCapsule, and moves it to another spot instance, where the job will continue. In fact, the application is unaware of the floating event. This use case alone, we have found, consistently saves our users more than 60% in compute cost.Submitting a job with the Float GUI<img src="https://miro.medium.com/v2/resize:fit:1400/0*yALS-Fy-kHvybR0h.png" height="502" width="700">Launching your job in the Cloud takes &lt;1 minuteAnother use case is rightsizing. Often an application’s resource utilization curve has peaks and valleys over time. That is problematic for picking the appropriate machine capacity for the application. However short-lived, the peak memory usage typically becomes the minimum memory capacity for the machine, lest the application will experience an OOM and crash. That also means that during the majority of the execution, the machine is underutilized.By making the application runtime float on the cloud VM, Memory Machine Cloud has a great solution for this problem. It constantly monitors the resource utilization of the application, and when it detects under- or over- utilization conditions, it can “float” the application to a more appropriate machine. This becomes especially handy when the user uses a workflow manager, such as NextFlow or Cromwell, to kickoff hundreds and thousands of jobs, and it is virtually impossible to manually pick the most appropriate machine types for each of the jobs.Monitoring resource utilization and “floating” the app to a more appropriate VM<img src="https://miro.medium.com/v2/resize:fit:1400/0*xs9uaSJDYLK_n3m2.png" height="361" width="700">WaveWatcher shows how jobs automatically “surf” to the most appropriate instance type<h2>The post-server era</h2>In the hardware world, we are seeing tremendous progress in the deconstruction of servers, composable CPU, Storage, and most recently, memory. Those innovations will manifest themselves in the public cloud soon. There is a need and an opportunity to build the automation layer that enables application users and IT operators to transition into “post-server” mode TODAY, not only for cloud native applications, but for all applications. That is the mission of Memory Machine Cloud.<iframe class="ql-video ql-align-center" frameborder="0" allowfullscreen="true" src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2juNvP3BbK8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2juNvP3BbK8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F2juNvP3BbK8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" height="480" width="854"></iframe>

Blog - Reduce Cloud Cost