Posts

How to properly load balance your backup infrastructure

Veeam Backup & Replication is known for ease of installation and a moderate learning curve. It is something that we take as a great achievement, but as we see in our support practice, it can sometimes lead to a “deploy and forget” approach, without fine-tuning the software or learning the nuances of its work. In our previous blog posts, we examined tape configuration considerations and some common misconfigurations. This time, the blog post is aimed at giving the reader some insight on a Veeam Backup & Replication infrastructure, how data flows between the components, and most importantly, how to properly load balance backup components so that the system can work stably and efficiently.

Overview of a Veeam Backup & Replication infrastructure

Veeam Backup & Replication is a modular system. This means that Veeam as a backup solution consists of a number of components, each with a specific function. Examples of such components are the Veeam server itself (as the management component), proxy, repository, WAN accelerator and others. Of course, several components can be installed on a single server (provided that it has sufficient resources) and many customers opt for all-in-one installations. However, distributing components can give several benefits:

  • For customers with branch offices, it is possible to localize the majority of backup traffic by deploying components locally.
  • It allows to scale out easily. If your backup window increases, you can deploy an additional proxy. If you need to expand your backup repository, you can switch to scale-out backup repository and add new extents as needed.
  • You can achieve a High Availability for some of the components. For example, if you have multiple proxies and one goes offline, the backups will still be created.

Such system can only work efficiently if everything is balanced. An unbalanced backup infrastructure can slow down due to unexpected bottlenecks or even cause backup failures because of overloaded components.

Let’s review how data flows in a Veeam infrastructure during a backup (we’re using a vSphere environment in this example):

veeam 1

All data in Veeam Backup & Replication flows between source and target transport agents. Let’s take a backup job as an example: a source agent is running on a backup proxy and its job is to read the data from a datastore, apply compression and source-side deduplication and send it over to a target agent. The target agent is running directly on a Windows/Linux repository or a gateway if a CIFS share is used. Its job is to apply a target-side deduplication and save the data in a backup file (.VKB, .VIB etc).

That means there are always two components involved, even if they are essentially on the same server and both must be taken into account when planning the resources.

Tasks balancing between proxy and repository

To start, we must examine the notion of a “task.” In Veeam Backup & Replication, a task is equal to a VM disk transfer. So, if you have a job with 5 VMs and each has 2 virtual disks, there is a total of 10 tasks to process. Veeam Backup & Replication is able to process multiple tasks in parallel, but the number is still limited.

If you go to the proxy properties, on the first step you can configure the maximum concurrent tasks this proxy can process in parallel:

veeam 2

For normal backup operations, a task on the repository side also means one virtual disk transfer.

On the repository side, you can find a very similar setting:

veeam 3

For normal backup operations, a task on the repository side also means one virtual disk transfer.

This brings us to our first important point: it is crucial to keep the resources and number of tasks in balance between proxy and repository.  Suppose you have 3 proxies set to 4 tasks each (that means that on the source side, 12 virtual disks can be processed in parallel), but the repository is set to 4 tasks only (that is the default setting). That means that only 4 tasks will be processed, leaving idle resources.

The meaning of a task on a repository is different when it comes to synthetic operations (like creating synthetic full). Recall that synthetic operations do not use proxies and happen locally on a Windows/Linux repository or between a gateway and a CIFS share. In this case for normal backup chains, a task is a backup job (so 4 tasks mean that 4 jobs will be able to generate synthetic full in parallel), while for per-VM backup chains, a task is still a VM (so 4 tasks mean that repo can generate 4 separate VBKs for 4 VMs in parallel). Depending on the setup, the same number of tasks can create a very different load on a repository! Be sure to analyze your setup (the backup job mode, the job scheduling, the per-VM option) and plan resources accordingly.

Note that, unlike for a proxy, you can disable the limit for number of parallel tasks for a repository. In this case, the repository will accept all incoming data flows from proxies. This might seem convenient at first, but we highly discourage from disabling this limitation, as it may lead to overload and even job failures. Consider this scenario: a job has many VMs with a total of 100 virtual disks to process and the repository uses the per-VM option. The proxies can process 10 disks in parallel and the repository is set to the unlimited number of tasks. During an incremental backup, the load on the repository will be naturally limited by proxies, so the system will be in balance. However, then a synthetic full starts. Synthetic full does not use proxies and all operations happen solely on the repository. Since the number of tasks is not limited, the repository will try to process all 100 tasks in parallel! This will require immense resources from the repository hardware and will likely cause an overload.

Considerations when using CIFS share

If you are using a Windows or Linux repository, the target agent will start directly on the server.  When using a CIFS share as a repository, the target agent starts on a special component called a “gateway,” that will receive the incoming traffic from the source agent and send the data blocks to the CIFS share. The gateway must be placed as close to the system sharing the folder over SMB as possible, especially in scenarios with a WAN connection. You should not create topologies with a proxy/gateway on one site and CIFS share on another site “in the cloud” — you will likely encounter periodic network failures.

The same load balancing considerations described previously apply to gateways as well. However, the gateway setup requires an additional attention because there are 2 options available — set the gateway explicitly or use an automatic selection mechanism:

Any Windows “managed server” can become a gateway for a CIFS share. Depending on the situation, both options can come handy. Let’s review them.

You can set the gateway explicitly. This option can simplify the resource management — there can be no surprises as to where the target agent will start. It is recommended to use this option if an access to the share is restricted to specific servers or in case of distributed environments — you don’t want your target agent to start far away from the server hosting the share!

Things become more interesting if you choose Automatic selection. If you are using several proxies, automatic selection gives ability to use more than one gateway and distribute the load. Automatic does not mean random though and there are indeed strict rules involved.

The target agent starts on the proxy that is doing the backup. In case of normal backup chains, if there are several jobs running in parallel and each is processed by its own proxy, then multiple target agents can start as well. However, within a single job, even if the VMs in the job are processed by several proxies, the target agent will start only on one proxy, the first to start processing. For per-VM backup chains, a separate target agent starts for each VM, so you can get the load distribution even within a single job.

Synthetic operations do not use proxies, so the selection mechanism is different: the target agent starts on the mount server associated with the repository (with an ability to fail over to Veeam server if the mount server in unavailable). This means that the load of synthetic operations will not be distributed across multiple servers. As mentioned above, we discourage from setting the number of tasks to unlimited — that can cause a huge load spike on the mount/Veeam server during synthetic operations.

Additional notes

Scale-out backup repositorySOBR is essentially a collection of usual repositories (called extents). You cannot point a backup job to a specific extent, only to SOBR, however extents retain some of settings, including the load control. So what was discussed about standalone repositories, pertains to SOBR extents as well. SOBR with per-VM option (enabled by default), the “Performance” placement policy and backup chains spread out across extents will be able to optimize the resource usage.

Backup copy. Instead of a proxy, source agents will start on the source repository. All considerations described above apply to source repositories as well (although in case of Backup Copy Job, synthetic operations on a source repository are logically not possible). Note that if the source repository is a CIFS share, the source agents will start on the mount server (with a failover to Veeam server).

Deduplication appliances. For DataDomain, StoreOnce (and possibly other appliances in the future) with Veeam integration enabled, the same considerations apply as for CIFS share repositories. For a StoreOnce repository with source-side deduplication (Low Bandwidth mode) the requirement to place gateway as close to the repository as possible does not apply — for example, a gateway on one site can be configured to send data to a StoreOnce appliance on another site over WAN.

Proxy affinity. A feature added in 9.5, proxy affinity creates a “priority list” of proxies that should be preferred when a certain repository is used.

If a proxy from the list is not available, a job will use any other available proxy. However, if the proxy is available, but does not have free task slots, the job will be paused waiting for free slots. Even though the proxy affinity is a very useful feature for distributed environments, it should be used with care, especially because it is very easy to set and forget about this option. Veeam Support encountered cases about “hanging” jobs which came down to the affinity setting that was enabled and forgotten about. More details on proxy affinity.

Conclusion

Whether you are setting up your backup infrastructure from scratch or have been using Veeam Backup & Replication for a long time, we encourage you to review your setup with the information from this blog post in mind. You might be able to optimize the use of resources or mitigate some pending risks!


This article was provided by our service partner veeam.com

vsphere

Get your data ready for vSphere 5.5 End of Support

There have been lots of articles and walkthroughs on how to make that upgrade work for you, and how to get to a supported level of vSphere. This VMware article is very thorough walking through each step of the process.

But we wanted to touch on making sure your data is protected prior, during and after the upgrade events.

If we look at the best practice upgrade path for vSphere, we’ll see how we make sure we’re protected at each step along the way:

vSphere EOL

Upgrade Path

The first thing that needs to be considered is what path you’ll be taking to get away from the end of general support of vSphere 5.5. You have two options:

  • vSphere 6.5 which is now going to be supported till November 2021 (so another 5 years’ time)
  • vSphere 6.7 which is the latest released version from VMware.

Another consideration to make here is support for surrounding and ecosystem partners, including Veeam. Today, Veeam fully supports vSphere 6.5 and 6.7, however, vSphere 6.5 U2 is NOT officially supported with Veeam Backup & Replication Update 3a due to the vSphere API regression.

The issue is isolated to over-provisioned environments with heavily loaded hosts (so more or less individual cases).

It’s also worth noting that there is no direct upgrade path from 5.5 to 6.7. If you’re currently running vSphere 5.5, you must first upgrade to either vSphere 6.0 or vSphere 6.5 before upgrading to vSphere 6.7.

Management – VMware Virtual Center

The first step of the vSphere upgrade path after you’ve decided and found the appropriate version, is to make sure you have a backup of your vCenter server. The vSphere 5.5 virtual center could be a Windows machine or it could be using the VCSA.

Both variants can be protected with Veeam, however, the VCSA runs on a Postgres-embedded database. Be sure to take an image-level backup with Veeam and then there is a database backup option within the appliance. Details of the second step can be found in this knowledge base article.

If you’re an existing Veeam customer, you’ll already be protecting the virtual center as part of one of your existing backup jobs.

You must also enable VMware tools quiescence to create transactionally-consistent backups and replicas for VMs that do not support Microsoft VSS (for example, Linux VMs). In this case, Veeam Backup & Replication will use the VMware Tools to freeze the file system and application data on the VM before backup or replication. VMware Tools quiescence is enabled at the job level for all VMs added to the job. By default, this option is disabled.

vSphere EOL 02

You must also ensure Application-Aware Image Processing (AAIP) is either disabled or excluded for the VCSA VM.

vSphere EOL 03

Virtual Machine Workloads

If you are already a Veeam customer, then you’ll already have your backup jobs created and working with success before the upgrade process begins. However, as part of the upgrade process, you’ll want to make sure that all backup job processes that initiate through the virtual center are paused during the upgrade process.

If the upgrade path consists of new hardware but with no vMotion licensing, then the following section will help.

Quick Migration

Veeam Quick Migration enables you to promptly migrate one or more VMs between ESXi hosts and datastores. Quick Migration allows for the migration of VMs in any state with minimum disruption.

More information on Quick Migration can be found in our user guide.

During the upgrade process

As already mentioned in the virtual machine workloads section, it is recommended to stop all vCenter-based actions prior to update. This includes Veeam, but also any other application or service that communicates with your vCenter environment. It is also worth noting that whilst the vCenter is unavailable, vSphere Distributed Resource Scheduler (DRS) and vSphere HA will not work.

Veeam vSphere Web Client

If you’re moving to vSphere 6.7 and you have the Veeam vSphere Web Client installed as a vSphere plug-in, you’ll need to install the new vSphere Veeam web client plug-in from a post-upgraded Veeam Enterprise Manager.

vSphere EOL 04

More detail can be found in Anthony Spiteri’s blog post on new HTML5 plug-in functionality.

You’ll also need to ensure that any VMware-based products or other integrated products vCenter supports are the latest versions as you upgrade to a newer version of vSphere.

Final Considerations

From a Veeam Availability perspective, the above steps are the areas that we can help and make sure that you are constantly protected against failure during the process. Each environment is going to be different and other considerations will need to be made.

Another useful link that should be used as part of your planning: Update sequence for vSphere 5.5 and its compatible VMware products (2057795)

One last thing is a shout out to one of my colleagues who has done an in-depth look at the vSphere upgrade process.


This article was provided by our service partner : Veeam.com 

Tips to backup & restore your SQL Server

Microsoft SQL Server is often one of the most critical applications in an organization, with too many uses to count. Due to its criticality, your SQL Server and its data should be thoroughly protected. Business operations rely on a core component like Microsoft SQL Server to manage databases and data. The importance of backing up this server and ensuring you have a recovery plan in place is tangible. People want consistent Availability of data. Any loss of critical application Availability can result in decreased productivity, lost sales, lost customer confidence and potentially loss of customers. Does your company have a recovery plan in place to protect its Microsoft SQL Server application Availability? Has this plan been thoroughly tested?

Microsoft SQL Server works on the backend of your critical applications, making it imperative to have a strategy set in place in case something happens to your server. Veeam specifically has tools to back up your SQL Server and restore it when needed. Veeam’s intuitive tool, Veeam Explorer for Microsoft SQL Server, is easy to use and doesn’t require you to be a database expert to quickly restore the database. This blog post aims to discuss using these tools and what Veeam can offer to help ensure your SQL Server databases are well protected and always available to your business.

The Basics

There are some things you should take note of when using Veeam to back up your Microsoft SQL Server. An important aspect and easy way to ensure your backup is consistent is to check that application-aware processing is enabled for the backup job. Application aware processing is Veeam’s proprietary technology based on Microsoft Volume Shadow Copy Service. This technology quiescences the applications running on the virtual machine to create a consistent view of data. This is done so there are no unfinished database transactions when a backup is performed. This technology creates a transactionally consistent backup of a running VM minimizing the potential for data loss.

Enabling Application Aware processing is just the first step, you must also consider how you want to handle the transaction logs. Veeam has different options available to help process the transaction logs. The options available are truncate logs, do not truncate logs, or backup logs periodically.

Figure 1: SQL Server Transaction logs Options

Figure 1 shows the Backup logs periodically option is selected in this scenario. This option supports any database restore operation offered through Veeam Backup & Replication. In this case, Veeam periodically will transfer transaction logs to the backup repository and store them with the SQL server VM backup, truncating logs on the original VM. Make sure you have set the recovery model for the required SQL Server database to full or bulk-logged.

If you decide you do not want to truncate logs, Veeam will preserve the logs. This option puts the control into the database administrator’s hands, allowing them to take care of the database logs. The other alternative is to truncate logs, this selection allows Veeam to perform a database restore to the state of the latest restore point. To read more about backing up transaction logs check out this blog post.

Data recovery

Veeam Explorer for Microsoft SQL Server delivers consistent application Availability through the different restore options it offers to you. These include the ability to restore a database to a specific point in time, restore a database to the same or different server, restore it back to its original location or export to a specified location. Other options include performing restores of multiple databases at once, the ability to perform a table-level recovery or running transaction log replay to perform quick point-in-time restores.

Figure 2: Veeam Explorer for Microsoft SQL Server

Recovery is the most important aspect of data Availability. SQL Transaction log backup allows you to back up your transaction logs on a regular basis meeting recovery point objectives (RPOs). This provides not only database recovery options, but also point-in-time database recovery. Transaction-level recovery saves you from a bad transaction such as a table drop, or a mass delete of records. This functionality allows you to do a restore to a point in time right before the bad transaction had occurred, for minimal data loss.

And it is available for FREE!

Veeam offers a variety of free products and Veeam Explorer for Microsoft SQL Server is one that is included in that bunch. If you are using Veeam Backup Free Edition already, you currently have this Explorer available to you. The free version allows you to view database information, export a database and export a database schema or data. If you’re interested in learning more about what you get with Veeam Backup Free Edition, be sure to download this HitchHikers Guide.

 


This article was provided by our service partner : veeam.com

veeam

Veeam Availability Suite 9.5 Update 3a is now available!

Platform support is a priority at Veeam. Whether that is the latest operating systems, new storage systems or updated hypervisors, we take platform support seriously. Since Veeam Backup & Replication 9.5 Update 3 has been released, a number of ecosystem changes have warranted an update ahead of the upcoming set of Veeam capabilities (due later this year) showcased at VeeamON. A larger update is coming soon, which is why we are referring to this release as Update 3a opposed to Update 4 (which is planned for later in the year). The main capabilities in this release are the new platforms supported as well as over 20 minor enhancements detailed in the KB article.

Update 3a will bring support for the latest VMware and Microsoft platforms that organizations need from Veeam. The list of new platforms supported by Veeam Backup & Replication are:

  • VMware vSphere 6.7
  • VMware vCloud Director 9.1
  • Preliminary support for VMware vSphere 6.5 U2  (See more below)
  • Microsoft Windows Server 1803
  • Microsoft Windows Hyper-V Server 1803
  • Microsoft Windows 10 April 2018 Update

There are supplemental platforms also supported in this update:

  • VMware Cloud on AWS version 1.3
  • Microsoft System Center Virtual Machine Manager 1801

This update is important as it means Veeam Backup & Replication will do the following:

  1. Install Veeam Backup & Replication 9.5 Update 3a on the new Windows operating systems
  2. Install components (such as proxies, repositories, etc.) on the new Windows operating systems
  3. Perform backup and replication jobs from the new vSphere platforms and the Hyper-V roles in the Microsoft Windows Server 1803 operating system

One different notation is the “Preliminary” support for VMware vSphere 6.5 Update 2. Those of you who have been following the weekly forum digest emails have additional insights to the many milestones that had to be achieved to get to this point. This is very important as with a product providing backup in the data center, we cannot take any risk of a false sense of security. These emails are also where you can get the latest from R&D on all the catch points that may arise; namely what we are seeing with vSphere 6.5 Update 2. Support for this release will likely come in an update to 6.5 Update 2 itself. The support statement is clarified well in this forum post, basically stating as it is there is a known issue a critical API for our use failing under load.

To remain on the cutting edge, many organizations like to maintain aggressive policies on upgrading to the latest vSpherevCloud DirectorWindows 10 and Windows Server releases; and ensuring that these platforms are supported for backup should be an important consideration. This is yet another reason why Veeam continues to work hard to deliver updated platform support as soon as possible. As you plan your next moves for your business, you can know that the platform support needed to keep those applications, systems and data available will be there with Veeam.


This article was provided by our service partner : veeam.com 

veeam

Veeam Availability Console U1 is now available

Managed service providers (MSPs) are playing an increasingly critical role in helping businesses of all sizes realize their digital transformation aspirations. The extensive offerings made available to businesses continue to allow them to shift day-to-day management onto you, the MSP, while allowing them to focus on more strategic initiatives. One of the most notable services being backup and recovery.

We introduced Veeam Availability Console in November 2017, a FREE, cloud-enabled management platform built specifically for service providers. Through this console, service providers can remotely manage and monitor the Availability of their customer’s virtual, physical and cloud-based workloads protected by Veeam solutions with ease. And, in just a few short months, we’ve seen incredible adoption across our global Veeam Cloud & Service Provider (VCSP) partner base, with overwhelmingly positive feedback.

Today, I’m happy to announce the General Availability (GA) of Veeam Availability Console U1, bringing with it some of the most hotly requested features to help further address the needs of your service provider business.

Enhanced Veeam Agent support

The initial release of Veeam Availability Console was capable of monitoring Veeam Agents deployed and managed by the service provider through Veeam Availability Console. New to U1 is the ability to achieve greater insights into your customer environments with new support that extends to monitoring and alarms for Veeam Agents that are managed by Veeam Backup & Replication. With this new capability, we’re enabling you to extend your monitoring services to even more Veeam customers that purchase their own Veeam Agents, but still want the expertise that you can bring to their business. And yes, this even includes monitoring support for Veeam Agent for Linux instances that are managed by Veeam Backup & Replication.

New user security group

VCSP partners wanting to delegate Veeam Availability Console access without granting complete control (like local administrator privileges) can now take advantage of the new operator role. This role permits access to everything within Veeam Availability Console essential to the remote monitoring and management of customer environments (you can even assign access to your employees on a company-by-company basis), but excludes access to Veeam Availability Console server configuration settings. Now you can assign access to Veeam Availability Console to your staff without exposing settings of the Veeam Availability Console server.

ConnectWise Manage integration

We’re introducing native integration with ConnectWise Manage. Through this new, seamless integration (available in the plugins library tab), the management, monitoring and billing of Veeam Availability Console-powered cloud backup and Disaster Recovery as a Service (DRaaS) can now be consolidated with your other managed service offerings into the single pane of glass that is ConnectWise Manage. This integration makes it easier and more efficient to expand your services portfolio while making administration of multiple, differing managed services much more efficient.

Matt Baldwin, President of Vertisys said, “This integration is exactly what my business needs to streamline our managed backup and DRaaS offering. The interface is clean and intuitive with just the right number of features. We project a yearly savings of 50 to 60 hours.”

Let’s take a closer look at some of the integration points between Veeam Availability Console and ConnectWise Manage.

Mapping companies

Firstly, the integration will help avoid a lot of manually intensive work by automatically synchronizing and mapping companies present in ConnectWise Manage with those in Veeam Availability Console. Automatic mapping is achieved through the company name. Before mapping is fully-complete, Veeam Availability Console allows you to check over what it’s automatically mapped before committing to the synchronization. If no match is found, mapping can be completed manually to an existing company or through the creation of a new company, with the option to send login credentials for the self-service customer portal, too.

Ticket creation

The integration also enables you to more quickly resolve issues before they impact your customers’ business through automatic ticket creation within ConnectWise Manage from Veeam Availability Console alarms. You can specify from the list of available alarms within Veeam Availability Console all those that are capable of triggering a ticket (e.g. failed backup, exceeding quota, etc.), and to which service board within ConnectWise Manage the ticket is posted. We’ve also enabled you with the capability to set delays (e.g. 1 minute, 5 minutes, 15 minutes, etc.) between the alarm occurring and the ticket posting, so issues like a temporary connectivity loss that self-resolves doesn’t trigger a ticket immediately. Every ticket created in ConnectWise Manage is automatically bundled with the corresponding configuration, such as representing a computer managed by Veeam Availability Console. This makes it incredibly easy for support engineers to find which component failed and where to go fix it. The integration also works in reverse, so that when tickets are closed within ConnectWise Manage, the corresponding alarm in Veeam Availability Console will be resolved.

Billing

The final part of the integration extends to billing, reducing complexities for you and your customers by consolidating invoices for all the managed services in your portfolio connected to ConnectWise Manage into a single bill. Not only this, but the integration allows for the automatic creation of new products in ConnectWise Manage, or mapping to existing ones. Service providers can select which agreement Veeam Availability Console-powered services should be added to on a per-customer basis, with agreements updated automatically based on activity, quota usage, etc.

Enhanced scalability

Finally, we’ve enhanced the scalability potential of Veeam Availability Console, enabling you to deliver your services to even more customers. The scalability improvements specifically align to the supported number of managed Veeam Backup & Replication servers, and this is especially useful when paired with the enhanced Veeam Agent support discussed earlier. This ensures optimal operation and performance when managing up to 10,000 Veeam Agents and up to 600 Veeam Backup & Replication servers, protecting 150-200 VMs and Veeam Agents each.


This article was provided by our service partner : veeam.com

Disaster Recovery Planning

How to build a disaster recovery plan with Veeam

Here’s a true story from one of our customers. A gas explosion resulted in a major power failure downtown, which in turn left the company’s primary data center offline for a week. This is a classic example of an IT Disaster – unexpected and unpredictable, disrupting business continuity and affecting Always-On operations. We can only imagine how much it could cost that company to stay offline for a week (as much as losing their business, I’d say), if they didn’t have a reliable disaster recovery plan and an Availability solution to execute this plan.

A solid disaster recovery plan makes your company resilient to IT disruptions and able to restore your services in case of disaster with minimal to no impact on users and business operations. It’s not just making regular backups, but a complex IT infrastructure assessment and documenting (including hardware, software, networks, power and facilities), business impact analysis of applications and workloads and planning on staff, roles and risk assessment. And above all, there’s an essential testing and exercising of your disaster recovery plan. If you don’t test, how would you know that it works as expected?

Unlike physical infrastructures with all their complexity, virtualization gives more flexibility in management and processes allowing you to do more with less. For virtualized data centers, Veeam delivers joint capabilities of enabling data Availability and infrastructure management. By using Veeam Availability Suite, you cover multiple points in your DR plan at once and get:

  • Offsite replication with traffic optimization and advanced capabilities
  • Easier disaster recovery orchestration and recovery testing
  • Infrastructure assessment and documentation
  • Capacity planning and “What if” modelling
  • Backup and virtual infrastructures monitoring and reporting

These also address compliance audit needs by providing you with up-to-date information on backed-up workloads, backups reliability and actual data recovery time versus your SLAs. If staying compliant and ready for audits is important for you, I recommend you read the new white paper by Hannes Kasparick, Mastering compliance, audits and disaster recovery planning with Veeam.

Replication as a core disaster recovery technology

DR planning includes defining the lowest possible RTO to minimize the disruption of business operations. In terms of ability to restore failed operations in minutes, replication mechanism wins the game allowing you to instantly switch the failed workload to its ready-to-use “clone” to get the lowest-possible RTO. For DR purposes, standby replicas of production VMs are stored on a remote secondary site or in the cloud. Even if the production site goes down, like in my example with a major power failure, a remote site remains unaffected by the disaster and can take the load.

Test your disaster recovery plan!

All data security and management standards (ISO family is not an exception) imply DR plan testing as a mandatory exercise. You can never know if everything will work as expected in cases of real disasters until you try it and run the planned procedures in advance. DR simulation will also allow you to ensure that your personnel are well-prepared for extreme IT situations and everyone mentioned in your DR plan is aware of the activities they need to perform. If you discover any drawbacks during DR testing – either human or software-related – you’ll have a good chance to fix your DR plan accordingly and thus potentially avoid serious disruptions in your business continuity.

Automated recovery verification for backups and replica restore points built in Veeam Backup & Replication (for no additional fees!) will save you much time and additional resources for testing. SureReplica allows to boot replicated VMs (VMware only for v9) to the necessary restore point in an isolated Virtual Lab and automatically perform heartbeat, ping and application tests against them. Also, you have an option to run your own customized tests – all without any impact on your production.

Final word

Disaster recovery planning is not just another bureaucracy, but a set of measures to maintain an organization’s business continuity. Built in compliance with international regulations and standards, a DR plan gives your customers a high level of confidence in your non-stop services, data security and Availability. Veeam helps you to stay compliant with both internal and external IT regulations, be ready for audit and be able to restore any system or data in minutes.


This article was provided by our service partner : veeam.com

veeam

How to avoid typical misconfigurations when setting up Veeam

This article is aimed at giving you a smooth start with Veeam Backup & Replication. It includes some basic advice on the initial setup, and outlines the most common misconfigurations that we, at Veeam Support, find in clients’ infrastructures during our investigations.

Recommendations on Veeam backup modes

In most cases, forward incremental or forever forward incremental backup modes are recommended as the fastest ones. Forever forward incremental (no periodic full backup) requires less space and offers decent performance. Forward incremental requires more space, but is also more robust (because a backup chain is further divided in subchains by periodic full backup).

Reverse incremental backup method is our oldest backup method and consequently the slowest. Depending on the type of storage in use, it can be three or more times slower than other modes. With the reverse incremental backup, you get a full backup as the last point in the chain. This allows for faster restores in case the most recent point is used, but the difference is often negligible in comparison to a forward incremental chain (if its length is not unreasonably long, we usually suggest it to be around 30 days).

Insights on the full backup

Synthetic full operation builds a full backup file from the restore points already residing in your repository. However, not every storage type provides a good performance with synthetic operations, so we advise to use active full backup as an alternative.

When you set up a synthetic full backup mode, there is an additional “Transform previous backup chains into rollbacks” option available. Keep in mind though that this option starts a task of transforming incremental backups (.VIB) into rollbacks (.VRB), which is very laborious for your target backup repository. For example, it will help you transform your current chain into the reverse incremental one for archival purposes. However, if you use it as a main backup method, it would produce a very specific backup chain consisting of a full backup file and a mix of forward and reverse incremental restore points.

Veeam - A forward incremental backup job with periodic synthetic full

Figure 1. A forward incremental backup job with periodic synthetic full.

Guest processing tips

Guest processing is used to create consistent backups of your VMs. And if they run instances of Microsoft Exchange, Active Directory, SharePoint, SQL Server and Oracle applications, you will be able to leverage granular restores using Veeam Explorers. Please note that guest processing relies on a VSS framework (a Windows feature), which should be functioning correctly, otherwise your backup jobs will fail.

To enable guest processing, go to Guest Processing of backup job properties. You should enable “Application-aware processing” option and you should provide an administrative account under guest OS credentials.

Figure 2. Guest processing step controls application-aware processing and indexing.

If some of VMs in the job require specific credentials, you can set them by clicking on the “Credentials” button. This brings up the Credentials menu. Click on “Set User…” to specify the credentials that should be used with the VM.

Veeam - Credentials menu

Figure 3. Credentials menu allows to set up users for each VM in the job.

Clicking on the “Applications…” button brings up a menu where you can specify options for supported applications and disable the guest processing for certain VMs, if needed.

Veeam - guest processing

Figure 4. In Applications menu, you can specify options for various application or disable guest processing completely for a VM.

VM guest file system indexing

With “VM Guest File System Indexing” enabled, Veeam Backup & Replication creates a catalog of files inside the VM, allowing you to use guest file search and perform 1-click restores through our Veeam Backup Enterprise Manager.

In case you don’t use the Enterprise Manager, then you can cut some (sometimes significant) time off your backup window and save space on the C: drive of a Veeam server by disabling this option. It doesn’t affect your ability to perform file level restores from your Veeam Backup & Replication console.

Secondary backup destination

No storage vendor can guarantee an absolute data integrity. Veeam checks a backup file once it’s written to a disk, but, with millions of operations happening on the datastore, occasional bits may get swapped causing silent corruption. Veeam Backup & Replication provides features like SureBackup and health checks that help detect an early corruption. However, sometimes it may be already too late, so it’s absolutely necessary to follow the 3-2-1 rule and use different sets of media in several locations to guarantee data Availability.

To maintain the 3-2-1 rule, right after creating a primary backup job, it’s advised to set up a secondary copy job. This can be a Backup Copy Job to a secondary storage, Backup Copy Job to a cloud repository or a copy to tape.

Instant VM recovery as it should be

Instant VM Recovery allows you to start a VM in minimal time right from a backup file. However, you need to keep in mind that a recovered VM still sits in your backup repository and consumes its resources. To finalize the restore process, the VM must be migrated back to the production. Too often we at Veeam Support see critical VMs working for weeks in the Instant VM Recovery mode until a datastore fills up and data is lost.

For those of you looking for a deep dive on the topic, I recommend the recent blog post on Instant VM Recovery by Veeam Vanguard Didier Van Hoye.

Veeam - VM Recovery mode

Figure 5. Soon after VM is started in the Instant VM Recovery mode you should initiate its migration back to the production.

Mind the CIFS as a main target repository

Veeam is storage agnostic and supports several types of backup repositories. Over the years, it was proven that a Windows or Linux physical server with internal storage gives the best performance in most cases.

Backup repository on a CIFS share still remains a popular choice, yet it generally offers the poorest performance of all options. Many modern NAS devices support iSCSI, so a better choice would be to create an iSCSI disk and present it to a Veeam server/proxy. Note though, that it’s also not recommended to use reverse incremental backup mode for repositories on NAS because it puts heavy IO load on the target.

Target proxy for replication

When replicating over the WAN, it is advised to deploy a backup proxy on the target site and configure it as a target proxy in replication job settings. This will create a robust channel between the two sites. We recommend setting a target proxy to NBD/Network mode, as using hot-add for replica can cause stuck and orphaned snapshots.

Note that when using WAN accelerators, a target proxy should still be deployed. Target WAN accelerator and target proxy can be installed on different or on a single machine, given it has enough resources.

Veeam - For replication over WAN, you should specify source and target proxy

Figure 6. For replication over WAN, you should specify source and target proxy.

Veeam - Set the target proxy mode to Network.

Figure 7. Set the target proxy mode to Network.

 

A must-do for a tape server

Tape server is a component responsible for communication with a tape device. It is installed on a physical machine to which a tape device is connected (“pass through” connections via ESXi host to a virtual machine are not supported!).

Veeam Backup & Replication gets the information about the library from the OS, so you should make sure that the latest drivers are installed and the tape device is visible correctly in the device manager.

You can find more info on using tapes with Veeam Backup & Replication in the previous blog post.

Watchdog your entire IT environment with Veeam ONE

With data becoming so valuable these days, organizations can’t afford to have their IT systems unavailable even for minutes. Monitoring your environment plays a key role to ensure Availability. You need to be alerted when things could go wrong and when it’s time to fix issues before they negatively impact your business. Veeam ONE does just that, not only for your VMware vSphere and Microsoft Hyper-V virtual environments, but also integrates with Veeam Backup & Replicationand, starting with Update 3, provides visibility for Veeam Agents as well.

In this blog post, I will talk about some of the Veeam ONE capabilities that will help you keep an eye on your IT systems.

Categorizing your infrastructure objects

Veeam ONE helps you categorize objects inside your infrastructure by business unit, department, purpose or SLA by means of its Business View component. This business categorization is integrated with Veeam ONE Monitor, enabling you to monitor, troubleshoot and report on business groups of VMware and Hyper-V objects.

Veeam ONE

The Configuration tab allows you to configure the basic application settings

After adding business categories into Veeam ONE, you can start monitoring your business groups through the Business View tab in Veeam ONE Monitor.

Business View in Veeam ONE Monitor, showing a virtual machine in the VMs with Snapshots category

You can also build reports for specified categories of objects. In the Workspace view of Veeam ONE Reporter, when you select a report, you can either choose to get details on the entire virtual environment, or on specific business view objects. For example, if you group VMs by department, you can create reports for a specific department in your organization.

The Business View objects window in Veeam ONE Reporter

Using alarms

There are predefined alarms in Veeam ONE for VMware vSphere and vCloud Director, Microsoft Hyper-V, Veeam Cloud Connect, Veeam Agents, and for internal Veeam ONE issues. All of them are designed to alert IT admins when any notable events or issues occur in their environment. With these alarms, you can easily identify, troubleshoot issues and quickly act to keep business operations running.

If they are used properly, alarms can be a critical method to notify you about the performance of your virtual environment. It’s important to know that too many alarms can lead you to ignore them, while too sensitive triggers can lead to false alarms.

The predefined alarms are built on best practices thresholds and trigger when the parameters defined in the alarm are different against collected data from the virtual and backup servers.

 

Alarm details in the Alarm Management tab. You can create custom alarms for any kind of tasks and events

When an alarm is triggered, the Veeam ONE console displays details about the root cause of the issue and some ways of resolution. Veeam ONE alarms are customizable and you can edit them to fit your business needs by adding different rules, changing thresholds or assigning them to different objects. You can also choose to send an email to a group when alarm’s severity changes or to suppress an alarm during scheduled activities. Moreover, in the Knowledge base tab you can add custom text to help you with solving the alarm next time it’s triggered.

Alarm Settings

Performance and health state monitoring

Veeam ONE enables you to monitor the performance of your VMware vSphere or Microsoft Hyper-V environment through comprehensive charts. In Infrastructure View, you can find information about the latest alarms, CPU, memory and disk resources for different timeframes, as well as network usage details. As you browse in the Infrastructure tree, the Summary tab will display different information for different objects and the rest of the tabs will vary too as you move deeper in the environment.

If the parent object is selected, the Infrastructure Summary tab will display the health state overview, including the Host State, Datastores State and the Virtual Machines state.

Along the navigation menu, Veeam ONE Monitor also provides details about Alarms (as you saw earlier in this post), as well as very well-organized metrics on resource consumption. Here you can check the VMs and hosts that use the most resources (CPU, memory, network usage and more), but also the hosts with the lowest load. These stats are available for both VMware vSphere and Microsoft Hyper-V environments.

The CPU chart shows the amount of used processor resources on a machine where a backup infrastructure component runs. Graphs in the CPU chart illustrate the level of processor usage for every separate CPU on the machine. The Total graph shows the cumulative processor utilization for all CPUs.

Capacity planning

While Veeam ONE Monitor provides extensive visibility over your IT infrastructure, the Reporter makes it easy for IT administrators to obtain detailed analysis of their virtual infrastructure and helps to take the guesswork out of capacity planning.

In virtual environments, capacity planning is a critical task for housekeeping and allows IT admins to make correct decisions based on accurate forecasts. The first thing we can see in Veeam ONE Reporter is the VMware Capacity Planning dashboard, which includes few widgets that offer a sneak peek into the more detailed reports of the infrastructure.

Veeam ONE 07

The VMware Capacity Planning dashboards displays details about Hosts and Clusters, Datastores, loss of a host and how many VMs can be added to the infrastructure without having to purchase more resources

Now let’s take a look at the capacity planning reports that we have available: Capacity Planning, Host Failure Modelling, How Many More VMs Can be Provisioned and Over-provisioned Datastores.

The reports are based on collected and analyzed historical data and they are very customizable, allowing you to choose individual or all datastores, set limits for CPU and Memory, select timeframe of the analyzed performance data, as well as the period of planning.

Veeam ONE 08

The Capacity Planning report provides very detailed forecast on my virtual infrastructure

At the end of the day, the capacity planning exercise helps you answer some simple questions: Do I have enough resources for more virtual machines? When will I run out of resources? Don’t underestimate these questions, they may save you time and money one day.

Agents monitoring and reporting

Update 3 came with many new features for Veeam Backup & Replication 9.5 and Veeam ONE 9.5, and one of the most highly-anticipated is agent management. Veeam ONE 9.5 now supports agent monitoring and reporting for Veeam Agent for Microsoft Windows and Veeam Agent for Linux, enabling you to have visibility into both your virtual and physical infrastructures.

You can have real-time monitoring and alerting for the Veeam Agent jobs managed by Veeam Backup & Replication servers that you monitor in Veeam ONE. To do so, go to Data Protection view in Veeam ONE Monitor and open the Agent Jobs tab of the desired backup infrastructure node. For each job, you will gain information such as the status of a backup job (Success, Warning, Failed, Running, or jobs with no status), backup job name, type, transferred data and more.

Veeam ONE 09

Veeam Agent jobs in Veeam ONE Monitor

In the Veeam Backup Agents report you can choose to include business groups (defined in Veeam ONE Business View) or Veeam Backup & Replication servers and protection groups. Likewise, you can choose to include in the report either specific Veeam backup agents or backup jobs and policies.

Veeam ONE 10

Selecting the report’s parameters

Veeam ONE 11

Veeam ONE 12

In my example, I chose to have a report for Windows-only machines with the RPO (Recovery Point Objective) of 1 week, meaning I will get a list of computers protected weekly. On the second page, you’ll find a detailed list of protected and unprotected computers, including information like IP address, Backup Job/Policy, Last Backup Date and more.

Conclusion

Monitoring your entire IT infrastructure can often be overlooked. This is a mistake that can not only cost your business money, but also its reputation. Minor breaches, due to their complexity, can cause performance issues in virtual and physical environments. However, this can be avoided with a good monitoring tool that alerts you when things are not working as they should. Veeam ONE provides a comprehensive set of alerts and a very user-friendly interface that facilitates visibility, troubleshooting and resolving issues.


This article was provided by our service partner Veeam

MySQL

Veeam – Linux VM: A place to back up MySQL

What does it take to back up MySQL on a Linux VM? This is a riddle we sometimes hear at Veeam: When running on a Linux VM, how does one quiesce MySQL databases? Unfortunately, there are not many new ways to answer this riddle, and the answers we currently have are already tried and tested!

The answers can be found in our popular white paper Consistent protection of MySQL/MariaDB with Veeam, written by Solutions Architect Pascal Di Marco. The paper is available for download on our website and describes three different methods for backing up MySQL/MariaDB on a Linux VM. Two hot backup methods running pre- and post-snapshot scripts, and cold backup using database shutdown. This makes use of VMware tools installed on the Linux VM. It is not straightforward like a Microsoft SQL quiescence, because Linux doesn’t have a VSS mechanism like Windows does.

Described are methods of backing up MySQL/MariaDB on a Linux VM using activated scripts local to the database. VMware can run a script to act before the snapshot is created, known as the pre-freeze script and can run a script to do things after the snapshot is created, known as the post-thaw script.

Here’s a quick summary:

Option 1: Hot backup — Database online dump

The mysqldump command copies a database to storage accessible from the MySQL server, taking an online dump of each database without disrupting the MySQL service. This method lets you take a transaction consistent backup of databases but more steps are needed to perform a restore. As with Option 1, the pre-freeze-script will only run if you have the VMware tools running.

Advantage: This allows for 100% uptime; the MySQL service does not stop and the dumped databases are in a transaction-consistent state.

Disadvantage: Depending on the size of your databases, the process may take a considerable amount of time to achieve. A second copy of the database means extra storage space is required to maintain it.

Option 2: Hot backup — Database freezing

Stop the MySQL service for a few moments while the snapshot is created, then start it again. The post-thaw script will not run until the snapshot is created. The pre-freeze script and post-thaw script will only be able to run if you have the VMware tools running in your MySQL server.

Advantage: This is quick and simple, allowing you to take a transaction of all databases with no additional disk usage local to the MySQL server.

Disadvantage: Databases running on the MySQL server will briefly be unavailable, and applications that need 100% uptime may not find this suitable.

Option 3: Cold Backup — Database shutdown

In this method, the application service will be stopped during snapshot creation and restarted once the VM snapshot has been created. It requires permission to start and stop application services but does not require MySQL user permissions. You can authenticate by either using the MySQL default configuration file or hardcoding the username and password in the script.

Advantage: This is easy to set up and doesn’t take extra space. It provides a short RTO, since no further action is required aside from booting the restored guest.

Disadvantage: The databases will be totally unavailable while the guest snapshot is created.

Recovery

Guest recovery: The cold backup and freeze method will leave the database consistent and able to start up without additional operation, so restoring the VM from the backup files is the only operation to perform. The guest recovery may benefit from Veeam’s Instant VM Recovery feature, which lets you boot up the guest directly from the Veeam Backup Repository in minutes.

Additional dump restoration: The extra task of injecting the dump file into the database using file redirection is necessary if the following is true: The issue is not limited to a database outage, the entire VM must be recovered from the Veeam Backup file and the database dump method has been used.

Veeam U-AIR database restoration: Whether it is a granular or a full database restoration, Veeam U-AIR wizard can be used in conjunction with any relevant database management tool such as MySQL Workbench to recover a database item.

 

veeam

Getting started with Veeam Explorer for Microsoft SQL Server

Believe it or not, I used to work a lot with Microsoft SQL Server. While I did not call myself a database administrator (DBA), I did know my way around a database or two. Since I’ve been at Veeam, I have always enjoyed telling the Veeam story around using SQL Server as a critical application that needs the best Availability options.

That’s why I took particular interest in Veeam Explorer for Microsoft SQL Server that came in Veeam Backup & Replication. Veeam Explorer for Microsoft SQL Server allows application-specific restores of SQL databases, and also contents of tables, objects such as stored procedures, views and more. Additionally, you can also restore the databases to a specific transaction.

This is a great combination of functionality from the established application-aware image processing with a dedicated tool for database restores in Veeam Explorer for Microsoft SQL Server. Additionally, Veeam Backup & Replication and the Veeam Agent for Microsoft Windows also provide an image backup of the entire system.

For those who are not a DBA, sometimes dealing with low-level SQL Server topics can be a bit overwhelming. To help this process, I created a few scripts to help individuals learn this type of interaction with SQL Server. I put three (and a deleted script) up on the Veeam Github site. To use this script, only an S:\ drive is needed (the path can be changed) to create the sample database and put in a SQL Server Agent job to automatically run a few stored procedures that will insert and delete random data. This creates a database called SQLGREENDREAM.

After running the three scripts to create the database, implement the random number function and set the schedule to create the random data (2 records) and delete 1 record. The SQL Server Transaction Log Backup will show the new database being backed up after the next incremental backup:

Veeam Explorer for Microsoft SQL Server

Once the interval of the SQL Server Agent job runs (12 minutes in the GitHub script) and the Veeam Backup Job interval passes, the most selective restore point option can be selected in Veeam Explorer for Microsoft SQL Server. This selective option, to restore to a specific transaction, is shown in the figure below:

Veeam Explorer for Microsoft SQL Server

Once the interval of the SQL Server Agent job runs (12 minutes in the GitHub script) and the Veeam Backup copy interval process through a time when the test data has been run, the restore to a specific transaction option can be visible to the controlled scripting for the SQLGREENDREAM database in the GitHub repository. Then you can see the records in question being just as scripted, 2 records added then one record deleted. Those entries are done by the SQL Server Agent:

From there, the restores can be done with confidence to see how the SQL databases are restored with Veeam. With the sample scripts in the GitHub repository, one can become more comfortable with these restore situations when venturing out of normal comfort zones! If you are using Veeam Backup Free Edition and the SQL Server is a VM being backed up, you can still use Veeam Explorer for Microsoft SQL Server to restore the database to the time of the image-based backup; just no transaction rollback. You can use the NFR program for a fully functional installation also.


This article was reposted from Veeam.com