Vast crypto mining data centers, consuming electricity at the rate of a small country while simultaneously discharging heat into the air, are now common pop culture references, resulting in a push to reduce crypto mining’s power requirements. For example, in 2022 [Ethereum](https://ethereum.org/en/roadmap/merge/) officially switched to a Proof-of-Stake (PoS) consensus mechanism as a more energy-efficient way (energy consumption decreased by about 99 percent) to validate transactions and add new blocks to the blockchain.

The power consumption of conventional data centers worldwide is comparable to crypto mining data centers, so the crypto mining image tends to envelop all data centers. Yet the two scenarios are very different. It is not possible to change a single algorithm to reduce the power consumption of a data center supporting a wide array of applications. In most cases, the biggest reduction in the carbon footprint associated with general purpose computing (web site hosting, database transactions, machine learning, and so on) can be achieved by moving from on-premises to the cloud.

![](https://baywatch-api.memverge.com/media/upload/202403/240310180436734115.png)


Source: [Ethereum](https://blog.ethereum.org/2021/05/18/country-power-no-more)



## The Benefits of Sharing Resources

The risk of trivializing the complexity of carbon footprint analysis notwithstanding, consider a simple analogy. Suppose your grocery store only sells milk in one-gallon jugs. Each week, you buy a gallon of milk. You consume half a gallon and by the end of the week, the remaining half gallon has soured, so you throw it out. Carbon footprint generated: one gallon of milk and one plastic jug.

It turns out your neighbor has the same pattern. You decide to buy one gallon of milk and share it. Carbon footprint generated per person: a half gallon of milk and half a plastic jug. That is a 50 percent reduction. The analogy can be extended further: suppose you switch from dairy to soy milk whose production releases lower levels of carbon emissions. That’s like switching to an energy source with lower carbon intensity. This is the cloud computing model.

![](https://baywatch-api.memverge.com/media/upload/202403/240310180503991053.jpeg)


Source: [Statista](https://www.statista.com/chart/22659/cows-milk-plant-milk-sustainability/)

## Operational and Embedded Emissions

Hyperscale Cloud Service Providers (CSPs) have invested vast sums in optimizing power and heat distribution in their data centers so that roughly 90 percent of the electricity drawn is used to power IT equipment directly. By using electricity from renewable energy sources and purchasing carbon offset credits, the hyperscale CSPs ([Google Cloud](https://www.google.com/about/datacenters/cleanenergy/), [AWS](https://sustainability.aboutamazon.com/climate-solutions/renewable-energy?energyType=true), and [Azure](https://azure.microsoft.com/en-us/explore/global-infrastructure/sustainability)) claim that they are either carbon neutral for their electrical power today or will get there soon. Furthermore, much of the heat is [captured](https://www.weforum.org/agenda/2022/08/sustainable-data-centre-heating/). So even though not all the power consumed in every data center is from renewable sources, the focus is shifting from [operational emissions](https://www.cloudcarbonfootprint.org/docs/methodology/#summary) to other generators of carbon emissions. 

### Carbon Credits: Do they work?

Carbon offset credits are a market-based mechanism for reducing global emissions. If a company cannot reduce its greenhouse gas emissions, it pays for the equivalent amount of greenhouse gas to be removed from (or not added to) the atmosphere. The challenge is in implementation because the quality of carbon offset credits varies. Poor quality carbon offsets ["over-credit"](https://climate.mit.edu/explainers/carbon-offsets) the impact, that is, the reduction in emissions is less than promised. Metrics, such as [additionality](https://www.offsetguide.org/high-quality-offsets/additionality/), [immediacy](https://mitsloan.mit.edu/ideas-made-to-matter/how-to-choose-carbon-offsets-actually-cut-emissions), and [leakage](https://doi.org/10.1016/j.oneear.2023.05.024), are used to verify whether the offset works as promised.

## Reducing Embedded Emissions

CSPs purchase servers, storage arrays and other IT equipment to stock their data centers. To the CSP, these are sources of embedded emissions ([Scope 3](https://ghgprotocol.org/scope-3-calculation-guidance-2)), that is, the manufacture and transport of this equipment has an associated carbon footprint. Utilization is the key metric in apportioning the embedded emissions to end users – just like the volume of milk consumed compared to the size of the milk jug.

A typical on-premises data center may achieve server utilization of around [20 percent](https://www.nrdc.org/energy/files/data-center-efficiency-assessment-IP.pdf). To consume all the compute cycles that a single server (running at 100 percent) could provide, the organization would need to purchase five servers. In contrast, server utilization in a CSP is at least [50 percent](https://aws.amazon.com/blogs/aws/cloud-computing-server-utilization-the-environment/), so the CSP would only need to purchase two servers. That is a 60 percent reduction in embedded emissions, although the power consumption of a server does rise as utilization increases.

## Managing VM Resources to Reduce Carbon Footprint

A modern data center server has, aside from power supplies, network interfaces, and storage devices, multiple CPU slots and multiple memory slots for each CPU slot. Each CPU comprises physical processing units (cores). For example, Intel’s third-generation Xeon Scalable processor, codenamed [Ice Lake](https://www.hpcwire.com/2021/04/06/intel-launches-10nm-ice-lake-datacenter-cpu-with-up-to-40-cores/), can contain up to 40 cores. The enabling technology for sharing server resources among end users is the hypervisor, which virtualizes the physical components into (ephemeral) virtual machines (VMs) of varying capacities (measured in terms of cores and memory).

The key to reducing carbon footprint (including embedded emissions if we assume operational emissions [[GoClimate](https://www.goclimate.com/blog/the-carbon-footprint-of-servers/), [Amazon](https://sustainability.aboutamazon.com/products-services/the-cloud?energyType=true)] are zero or close to zero in hyperscale CSPs) is managing VM resources to maximize utilization. That is, size the VM for the job it needs to execute and release the resources when they are no longer needed so that other users have access to the resources. Sophisticated checkpoint/restore mechanisms (like Memory Machine Cloud's WaveRider feature) can optimize VM instances even more – by migrating jobs in real time as resource demands change while the job is executing. If resources are idle, but locked up (not available to other users), the CSP is forced to deploy additional servers and incur additional carbon footprint. 

Suppose, for example, that server utilization increases from 50 to 65 percent because of intelligent VM management. This would decrease the carbon footprint (associated with the entire server) by 23 percent. For the end user who is tracking the Scope 3 emissions attributable to running a workload on a cloud server, the effect of continually rightsizing the VM as the workload executes can be significant. For example, if without rightsizing, an end user consumes 20 percent of a server’s resources and with rightsizing the end user consumes 10 percent of the server’s resources (averaged over the run), the embedded emissions attributable to the end user are reduced by 50 percent.

![](https://baywatch-api.memverge.com/media/upload/202403/240310180008526778.png)

Source: [Benefits of Virtualization](https://cs181cloudstorage.weebly.com/benefits-of-virtualization.html)

## Conclusion

Modern hyperscale data centers are engineering marvels that are key components in reducing global carbon emissions, but underneath is an old-fashioned idea: take only what you need and if you have extra, share it with your neighbors. It’ll save you money as well.

### What is application observability and why it is important

Application observability gains insights into the behavior of software applications utilizing the information from logs, metrics and traces. The goals include but not limited to

- identify bottlenecks and optimize performance
- identify issues and troubleshoot problems quickly
- identify resource requirements and determine the most appropriate corresponding infrastructure

### What is Memory Machine Cloud WaveWatcher and how it can help

The Memory Machine Cloud WaveWatcher service builds on its ability to track CPU, memory, network, storage (size, throughput, and IO) used by a ‘job’ (could be an application, a workflow, or a pipeline). Take the below WaveWatcher result of a Sentieon WGS pipeline as an example:

![WaveWatcher for Sentieon WGS Pipeline](https://baywatch-api.memverge.com/media/upload/202311/231120214738149768.png)

We can see how this pipeline can be characterized into different steps, each with corresponding ‘behaviors.’ The resources the steps consume are shown in the table below:
| Step | Function | Observability - Memory Usage | Observability - CPU Usage | Observability - Network | Observability - Storage Usage | Observability - Storage IO | What can / should we do? |
| - | - | - | - | - | - | - | - |
1 | Validate license and download reference and sample data | extremely low | extremely low | extensive | increasing | extensive | Use a less memory and CPU intensive but network and storage optimized instance |
2 | BWA-MEM | continuously growing | extremely high | none | increasing | medium | Use a memory and CPU optimized instance and larger instance
3 | Dedup | dynamically changing | dynamically changing | zero | dynamically changing | dynamically changing | Use a CPU optimized instance and larger instance |
4 | Variant calling, filtering and genotyping | high | extremely high | zero | retain | zero | Use a memory and CPU optimized instance and medium instance

### Go beyond observability - Memory Machine Cloud WaveRider

It is helpful to gain observability but the ultimate question is - what can be improved, and can it be automated? The data from WaveWatcher can then be used by the WaveRider service to automatically right-size resources, keep the application / pipeline / workflow state, and reduce CO2 footprint, hence to

- avoid under-provision which may result application performance bottleneck or even OOM, and
- avoid over-provision which spends unnecessary money and CO2 footprint.

More detailed Sentieon WGS pipeline benchmark can be found [here](https://www.mmcloud.io/blog/sentieon-genomics-tools-boosting-performance-wit-3).

### Three pillars of Memory Machine Cloud - observability (WaveWatcher), right-sizing (WaveRider), and job automation (Float)

Last but not the least, how to automate the whole process with minimum human intervention? Just like the three pillars of observability (logs, metrics and traces), there are three pillars for Memory Machine Cloud - job automation (Float), observability (WaveWatcher), and right-sizing (WaveRider).

- The user submits a job (for example, a bioinformatics analysis pipeline) via Float CLI or GUI.
- Memory Machine Cloud WaveWatcher automatically observes the behavior including the resources needed
- Memory Machine Cloud WaveRider automatically right-sizes / vertical scaling based on WaveWatcher’s observation supporting stateful job

These three pillars make a complete cloud automation solution removing the hassle of integrations among multiple solutions.

### Comparison of Memory Machine Cloud with modern solutions

Below is a summary table of comparison of Memory Machine Cloud with modern solutions, where Memory Machine Cloud provides an all-in-one solution.
| Category | Comparison | MemVerge MMCloud | Grafana | Datadog | Spot by NetApp | YellowDog | AWS CloudWatch | AWS Compute Optimizer | AWS Batch |
|:----------:|:---------------------------------------------------------------:|:------------------:|:----------:|:----------:|:--------------:|:---------------:|:--------------:|:---------------------:|:----------:|
| Feature | Observability | Yes (WaveWatcher) | Yes | Yes | Yes | Yes | Yes | Yes | No |
| Feature | Automatic right-sizing stateless application | Yes (WaveRider) | No | No | Yes | Yes | No | No | No |
| Feature | Automatic right-sizing stateful, non-fault-tolerant application | Yes (WaveRider) | No | No | No | No | No | No | No |
| Feature | Job automation | Yes (Float) | No | No | No | Yes | No | No | Yes |
| Deployment | On-prem version | Yes | Yes | No | No | Yes | No | No | No |
| Pricing | Pricing | All-in-one license | By license | By service | By license | Basic + add-ons | By service | Opt-in | By service |

\*note - more detailed comparisons with Spot by NetApp can be found [here](https://www.mmcloud.io/blog/memory-machine-cloud-vs-spot-by-netapp)

### Call for action - save your spending and time

Do you need to run stateful, non-fault-tolerant workloads or pipelines on the cloud? Would you like to benefit from automatic right-sizing and vertical scaling and reduce both cloud cost and job running hours without worrying about integrations among multiple solutions?  [Please try Memory Machine Cloud for free](https://www.mmcloud.io/customer/register).

High-performance computing (HPC) traditionally involves clusters of compute servers deployed in dedicated facilities. In recent years, demands for scale and performance have pushed HPC from on-premises to the cloud - a transition that offers new opportunities as well as significant hurdles.

On the positive side, the cloud offers virtually unlimited on-demand computing and storage resources because cloud platforms provide elastic capacity that can grow (and shrink) dynamically when resource demands change.

There are economic benefits to moving to the cloud, although not always. A stable workload that runs 24x7x365 for several years may be cheaper to run on dedicated hardware (cloud service providers have responded to this case by offering bare-metal instances). The pay-as-you-go model eliminates upfront capital expenses and avoids the costs associated with unused compute cycles. The range of compute instance types available in the cloud allows users to pay only for what they need, when they need it.

However, running applications in public clouds introduces new challenges. Each cloud service provider has an extensive catalog of services, and each service has a wide array of features. Without the knowledge required to configure and optimize cloud services, the researcher may encounter unpredictable or unsatisfactory application performance, unexpected costs, or unnecessary security risks.

This is where Memverge’s Memory Machine Cloud (MMCloud) can help. MMCloud is an easy-to-use cloud automation platform designed for scientific computing and data science. MMCloud simplifies the deployment of data-intensive pipelines and interactive computing applications in the cloud through automation and containerization.

Containers are the key technology that enables application mobility and automates the deployment of applications, at scale, in the cloud. The container includes all the software dependencies and runtime libraries needed to run the application in a reproducible way. But container deployment requires a cloud orchestration platform to schedule, instantiate, and manage containers. MMCloud provides an intuitive GUI (as well as a CLI) that allows the non-cloud specialist to deploy complex workflows as containerized applications in the cloud.

There are many reasons why scientists rely on MMCloud to enhance their research capabilities. Here are the top five reasons why you should choose MMCloud as the platform to run your cloud workloads.

**1. Seamless Expansion of High-performance Computing to the Cloud**

Hybrid cloud (a combination of on-premises and cloud resources) is a popular architecture for HPC because of its flexibility and scalability. When demand for computing resources exceeds what is available locally (on-premises), applications can “burst” to the cloud.

MMCloud effectively augments the local HPC environment with public cloud by enabling the user to launch jobs in the cloud using the same familiar HPC job managers such as SLURM and PBS. This way, scientists can operate in an efficient hybrid environment. Containerization and automation are a core feature of MMCloud. MMCloud automates deployment of containerized applications into the clouds, and it automatically deletes cloud resources after a job completes, removing the risk of unattended resources adding to the monthly cloud bill. Interactive sessions can be hibernated and restarted when needed again.

![Memory Machine Cloud HPC Bridge extends HPC into the cloud](https://baywatch-api.memverge.com/media/upload/202312/231207200802821427.jpeg)


Figure 1 MMCloud expands HPC to clouds

**2. Insightful Monitoring of Cloud Workloads**

Some users are reluctant to move workloads to the cloud because of a concern that they will have limited insight into, and control over, how their applications run in the cloud. Will the job complete in a shorter or longer time? Are the resources over- or under-provisioned?

Visibility into workload performance and resource utilization is critical when operating in the cloud. Granular monitoring of resource usage and costs can identify opportunities for optimization through rightsizing, shutting down idle resources or shifting workloads based on the current prices of compute instances. While cloud service providers do offer monitoring tools, these tools are not always easy to use and often come with additional costs.

MMCloud provides a comprehensive solution with advanced monitoring and metering capabilities, ensuring that scientists have a detailed understanding how their workloads run. CostReporter gives you a summary how much you spend on the jobs running in the cloud. WaveWatcher makes it easy to see the real-time consumption of resources including CPU, memory, storage, and network transfer speeds. For example, an application may exceed its storage limits or crash because of insufficient resources. Issues like these can be identified using MMCloud’s graphical interface, making troubleshooting much more straightforward.

MMCloud generates event and performance logs at both the job and host (worker node) level. Logs can be viewed in real-time to track the progress of a job as it executes, or downloaded after a job completes to identify issues or errors. The monitoring features enable researchers to fine-tune their environments for maximum efficiency.

![MMCloud WaveWatcher provides insights to cloud workloads](https://baywatch-api.memverge.com/media/upload/202311/231121040115584000.png)

Figure 2 MMCloud WaveWatcher provides insights to cloud workloads

**3. Reliable Spot Instances**

Almost all service providers offer spot Instances as a way for users to purchase excess computing capacity at a significantly lower price than on-demand instances (discounts can be as much as 90%). The trade-off is unpredictable availability. The cloud service provider can reclaim these instances with nominal warning (usually two minutes or less). Spot instances are ideal for stateless applications and can be used in stateful, batch-oriented pipelines if the workflow manager can requeue any failed jobs, although the total cost and wall clock time become unpredictable.

MMCloud's SpotSurfer feature allows stateful applications to always run to completion by "surfing" to a new VM instance if the underlying Spot Instance is reclaimed. Customers can realize the cost savings of spot instances without the risk of job interruptions. The technology that makes SpotSurfer possible is AppCapsule. AppCapsule creates a snapshot of the in-memory state as well as file system data. AppCapsule is triggered automatically when the cloud service provider signals that it is reclaiming the spot instance. Job execution pauses and then resumes on a new spot instance.

For interactive sessions, MMCloud provides a reverse proxy to ensure that each interactive session appears to clients as an IP address that does not change even if the session moves to a different virtual machine with a different IP address.
![MMCloud SpotSurfer protects workloads on spot instances](https://baywatch-api.memverge.com/media/upload/202311/231121040319580845.png)

Figure 3 MMCloud SpotSurfer protects workloads on spot instances

**4. Cost Control**

Cost management is a major concern for researchers, many of whom are grant-funded with strict budget constraints. While cloud services can be cost-effective, they can also lead to unpredictable expenses if not managed carefully. Institutions must develop new financial governance models to control cloud spending, ensuring that costs align with research budgets. Managing costs at cloud scale demands continuous monitoring of cloud resource usage to guard against runaway expenses. MMCloud’s Job Cost Reporting tools allow for precise budgeting, and the SurfZone feature provides real-time enforcement of limits on cloud spending.

Lifecycle management of cloud-based workloads can be critical for cost control. MMCloud provides control over of resources, for example, by enforcing automated schedules for job startup and shutdown, and maximum job execution times. This level of control is essential for managing costs in group settings where multiple users may run concurrent experiments. There are actions that individual users can take, for example, canceling jobs, moving jobs to smaller (cheaper) instances, or suspending interactive sessions. These actions can result in saving cost and can be used for avoiding idle charges.

Performance can be critical for cost control too. When jobs run on the spot instances, the slower jobs in data pipelines have a higher chance of getting reclaimed and the time and expense penalty for starting over might be huge. MMCloud’s integration with third-party high-performance file systems has improved the efficiency of file operation and has shown a significant advantage for Nextflow workflows.

WaveRadar, an interactive dashboard created on mmcloud.io is a free FinOps tool for viewing historical spot EC2 prices across all major AWS regions. WaveRadar lets you identify the regions offering the lowest rates for a given instance or instance family.

**5. Rightsizing Cloud Resources**

Compute instances available from cloud service providers range in size from one vCPU and 500 MB memory to 192 vCPUs and 1.5 TB memory (and even more in some specialized instances). Choosing the optimal cloud resources for scientific workloads is a complex task. Sizing for the peak load is safe but inefficient. Sizing for the average load reduces the cost but can lead to unacceptable performance or even out-of-memory (OOM) errors.

MMCloud's WaveRider feature simplifies this process by removing the need to select a single instance for each task. WaveRider provides the capability to migrate running jobs to optimally sized virtual machines based on their real-time resource utilization.

Job migration can be initiated in three ways: manually, driven by policy, or programmatically. At any time, you can manually migrate a job to a specific instance type, or to an instance whose capacities you define. With a rules-based policy, WaveRider monitors the resource utilization of your workload. If the upper threshold for CPU or memory utilization is crossed for a specified interval, the job is migrated to a virtual machine that has more virtual CPUs or more memory. Similarly, if the lower threshold is crossed and utilization remains low for a specified interval, the job is moved to a smaller virtual machine. For jobs with a well-known resource usage profile, job migration can be programmed into the job script so that the job migrates at predetermined stages in the execution.

Rightsizing has a positive effect on the optimal allocation and sharing of cloud resources. The optimization of using cloud resources has a profound impact on environmental sustainability. Users who proactively release underutilized resources allow the cloud service provider to increase server utilization. Fewer servers running at higher utilization lowers operational as well as embedded emissions. In a world increasingly conscious of climate change, optimizing cloud resource usage is a step towards reducing the overall carbon footprint of the IT industry. MMCloud provides a carbon emissions dashboard so that users become aware of the impact of how they run their jobs in the cloud.

![MMCloud Carbon Emission Monitor](https://baywatch-api.memverge.com/media/upload/202311/231121040303736482.png)

Figure 4 MMCloud Carbon Emission Monitor

**Conclusion**

Moving HPC workloads to the cloud, or bursting workloads to the cloud when needed, has benefits in terms of cost, scalability, efficiency, and sustainability. Challenges in usability and cost control in the cloud have prevented users from taking advantage of the cloud or a hybrid cloud architecture.

MMCloud is a powerful, yet easy to use, platform that facilitates the use of cloud computing by the research community. MMCloud is cloud-agnostic and is supported on AWS, Google Cloud, and AliCloud today. A free 30-day, no obligation trial is available. Register at [https://mmcloud.io/customer/register](https://mmcloud.io/customer/register) today.

Blog - Cloud Computing Sustainability