Spectrum Control Capacity Planning - IBM

© Copyright IBM Corporation 2017.

Spectrum Control Capacity Planning

Scott McPeek [email protected]

Washington Systems Center

Bryan Odom [email protected]

Software Advanced Technogoly (SWAT)

Washington Systems Center - Storage

© Copyright IBM Corporation 2017 Accelerate with IBM Storage.

• Capacity Planning defined; Capacity Planning strategies

• Fundamental questions that need to be answered

• Where to Start and Settings to Verify

• Key Capacity Planning Metrics and Reporting Options

• Spectrum Control Grouping Options with Alerts

• Identifying Waste, Inactive and Orphaned Storage

• Capacity Planning with Spectrum Virtualize and Spectrum Accelerate

• Putting it all together

1

Capacity Planning with Spectrum Control Agenda


© Copyright IBM Corporation 2017 Accelerate with IBM Storage. 2

Capacity Planning Defined:

Capacity planning is the process of determining the production capacity needed by an organization to meet changing demands for its products.

The phrase is also used in business computing and information technology as a synonym for capacity management. IT capacity planning

involves estimating the storage, computer hardware, software and

connection infrastructure resources required over some future period of time.

A common concern of enterprises is whether the required resources are

in place to handle an increase in users or number of interactions.

https://en.wikipedia.org/wiki/Capacity_planning



• Lead strategy is adding capacity in anticipation of an increase in demand

• Lag strategy refers to adding capacity only after the organization is

running at full capacity

• Match strategy is adding capacity in small amounts in response to

changing demand in the market

• Adjustment strategy is adding or reducing capacity in small or large

amounts due to consumer's demand, or, due to major changes to product

or system architecture.

3

Capacity Planning Strategies

The broad classes of capacity planning are lead strategy, lag strategy, match strategy, and adjustment strategy.



• Lead strategy is adding capacity in anticipation of an increase in demand

• This is the whole idea of “capacity planning”

• Requires historical metrics, reporting and monitoring to make it happen

• Lag strategy refers to adding capacity only after the organization is running at full capacity

• Nobody wants to be in this situation, scrambling to add capacity as quickly as possible

• Match strategy is adding capacity in small amounts in response to changing demand in the market

• Depending on the storage hardware, it may be possible to easily add incremental capacity

• Adjustment strategy is adding or reducing capacity in small or large amounts due to consumer's demand, or, due to major changes to product or system architecture

• Events outside of business as usual can introduce the need for additional capacity

• Examples would be a new system replacing an old system or ingesting an acquisition

4

Capacity Planning Strategies Put in the Storage Perspective



• What is the target utilization for storage?

• Can you track this utilization and be Alerted if projections start to deviate

• What is the time line for the capacity plan?

• Is 3 months too short, can we make a 12 month projection with confidence?

• What is the lead time required to add capacity?

• As in, if you need more capacity today, how long does it take before new capacity is installed and usable?

• If a surprise occurs, what can be done to “buy time”?

• Can you employ data reduction technologies in an emergency situation?

• Can you non-disruptively move capacity to alleviate constrained resources?

• Do performance requirements override capacity needs?

• Are there strategic directions that need to be kept in mind?

5

Capacity Planners Need To Know and Decide on some Fundamental Questions



• The first thing that must be decided is the metrics to be used for measurement

• We need to think in terms of useable capacity

• This means that the concept of Disks or unformatted space may not be the best option

• Pool Space on the other hand is really formatted space and is all useable capacity

• To make useable capacity work, all storage should be formatted into pool storage

• Some customers have “hidden” capacity as unformatted storage, which really distorts useable space over time

• In addition, hiding storage in unformatted capacity hinders overall performance

• The following chart is a good illustration of what happens to normalized metrics when

formatted pool storage is returned back to the storage system as unformatted space

• The charts from now on relate specifically to using Spectrum Control as a Capacity

Planning Tool

6

Where to Start?



Pool Capacity versus Allocated, Why not fully allocating storage make analysis difficult

Allocated Space does not really change

Physical Allocation % has a huge change

Pool Capacity decrease caused the Physical Allocation % Increase



While Data Reduction Technology has a huge impact on the amount of

data that can be stored, it does not negate the requirement to monitor the environment.

• Data Reduction Technology includes

• Thin Provisioning

• Compression

• Deduplificaiton

• Depending on the hardware, data reduction technologies are used in

conjunction with each other

• For the capacity planner, data reduction technology does not matter

• Capacity utilization will increase, even with data reduction technology in place

• This capacity must still be monitored and tracked over time

8

Data Reduction Technologies and Capacity Planning



• Capacity Planning is predicting future growth based on historical metrics

• Data Reduction technology adds more variables to growth prediction

• Will future Thin Provisioning ratios be the same as the past

• Will Compression ratios hold

• Will the next application’s Deduplication savings be the same as the past

• Taken together, the Capacity Planner must now not only predict how

storage needs are going to grow but also predict the data reduction

savings for future data to be stored

• What we will probably end up with is having to allow for a larger error

factor for future capacity needs

9

Data Reduction Technologies and Prediction



First Thing is to verify Capacity History Retention

Make sure that your History Retention matches your reporting requirements

Capacity History is not expensive in terms of repository storage, so go big



The above list is the epitome of TMI: Too Much Information

Capacity Planning Metrics – A Partial List

Storage system capacity metrics

• Allocated Space (GiB)

• Assigned Volume Space (GiB)

• Available Pool Space (GiB)

• Compression Savings (%)

• Data Deduplication Savings (%)

• Data Reduction (%)

• Physical Allocation (%)

• Pool Capacity (GiB)

• Pool Shortfall (%)

• Raw Disk Capacity (GiB)

• Total Volume Capacity (GiB)

• Unallocatable Volume Space (GiB)

• Unallocated Volume Space (GiB)

• Unassigned Volume Space (GiB)

• Virtual Allocation (%)

Pool capacity metrics

• Allocated Space (GiB)

• Assigned Volume Space (GiB)

• Available Pool Space (GiB)

• Available Repository Space (GiB)

• Available Soft Space (GiB)

• Capacity (GiB)

• Compression Savings (%)

• Data Reduction (%)

• Enterprise HDD Available Space (GiB)

• Enterprise HDD Capacity (GiB)

• Nearline HDD Available Space (GiB)

• Nearline HDD Capacity (GiB)

• Physical Allocation (%)

• Repository Capacity (GiB)

• Shortfall (%)

Pool cont’d

• Soft Space (GiB)

• SSD Available Space (GiB)

• SSD Capacity (GiB)

• Tier 0 Flash Available Space (GiB)

• Tier 0 Flash Capacity (GiB)

• Tier 1 Flash Available Space (GiB)

• Tier 1 Flash Capacity (GiB)

• Total Volume Capacity (GiB)

• Unallocatable Volume Space (GiB)

• Unallocated Volume Space (GiB)

• Unassigned Volume Space (GiB)

• Unused Space (GiB)

• Used Space (GiB)

• Virtual Allocation (%)



Spectrum Control Key Capacity Planning Metrics

Storage System Metrics Give Big Picture ViewsYou never want to be at “capacity” at the storage system level

• Physical Allocation %

• Shortfall %

Pool Metrics Generally Show a Subset of a Storage SystemLooking at pool consumption helps to identify which applications are causing growth

• Pool Capacity

• Allocated Pool Space

• Available Pool Space

• Virtual Allocation %

• Zero Capacity



Key Capacity Metrics to focus on: Physical Allocation %

Physical Allocation (%)

The percentage of physical capacity in the pools that is allocated to the regular volumes, the thin-provisioned volumes, and the volumes in child pools. Check the value for physical allocation to see:

• Whether the physical capacity of the pools is fully allocated. That is, the value for physical allocation is 100%.

• Whether you have sufficient capacity to provision new volumes with storage

• Whether you have sufficient capacity to allocate to the compressed and thin-provisioned volumes in the pools

Available for: All storage systems.



Storage System Physical Allocation % Sorted at the Washington Systems CenterThis view is of all Storage Systems, in reality we really do not care about them all.

Physical Allocation % for all Block Storage Systems



Physical Allocation % for Large Systems General Group

Here is a General Group which includes DS8000’s, XIV’s

and an A9000, sorted on Physical Allocation %.

Like most installations, we care about some devices more

than others, the DS8870 is getting worrisome.

We will talk more about grouping options later in this

presentation.



Key Capacity Metrics to focus on: Shortfall %

Shortfall (%)

The difference between the remaining unallocated volume space and the available capacity of the

associated pool, expressed as a percentage of the remaining unallocated volume space. The

shortfall represents the relative risk of running out of space for overallocated thin-provisioned

volumes. If the pool has sufficient available capacity to satisfy the remaining unallocated volume

space, no shortfall exists. As the remaining unallocated volume space grows, or as the available

pool capacity decreases, the shortfall increases and the risk of running out of space becomes

higher. If the available capacity of the pool is exhausted, the shortfall is 100% and any volumes that

are not yet fully allocated have run out of space.

If the pool isn't thin-provisioned, the shortfall percentage equals zero. If shortfall percentage isn't

calculated for the storage system, the field is left blank.

The following formula is used to calculate this value:

[(Unallocatable Space ÷ Committed but Unallocated Space) × 100]

You can use this percentage to determine when the amount of over-committed space in a pool is at

a critically high level. Specifically, if the physical space in a pool is less than the committed virtual

space, then the pool does not have enough space to fulfill the commitment to virtual space. This

value represents the percentage of the committed virtual space that is not available in a pool. As

more space is used over time by volumes while the pool capacity remains the same, this percentage increases.



Different Levels of Shortfall

Storage system

Pool

Volume



Shortfall % for all Block Storage Pools

This V9000 VersaStack Pool stands outWe should interrogate further



Pool Properties Detail to understand Shortfall %

34,776 GiB Volume Capacity

10,649 GiB Pool Capacity

327% Virtual Allocation



Key Capacity Metrics to focus on: Pool Capacity Metrics

Pool Capacity (GiB)

The total amount of storage space in the pool, which might include overhead space if the disks for the pool

aren't formatted. For XIV systems and IBM Spectrum Accelerate, capacity represents the physical or 'hard'

capacity of the pool, not the virtual 'soft' capacity.

Allocated Pool Space (GiB)

The amount of space that is allocated to the regular and thin-provisioned volumes in the pool. If the pool is a parent pool, the amount of space that is allocated to the volumes in the child pools is also included. The space that is allocated for thin-provisioned volumes is less than their virtual capacity, which is shown in the Total Volume Capacity column. If a pool does not contain thin-provisioned volumes, this value is the same as Total Volume Capacity.

Allocated space is the same as used space on all storage systems, except for storage systems that run IBM Spectrum Virtualize™. These storage systems might have more allocated space than used space if the storage administrator pre-allocated some space for thin-provisioned volumes when the volumes were created.

Available Pool Space (GiB)

The amount of space that is available to create new volumes in the pool. If the pool is a parent pool, the amount of space that is allocated to the volumes in the child pools is also included.



Pool Capacity, Allocated and Available Space for a V9000

Sorted on Available Pool Space



EasyTier Distributions – By Pool

Spectrum Control will give capacity information about EasyTier Pools. The

decisions to be made about how large to make each tier is complicated by the

need for performance. While it might make sense to add more capacity to a lower tier of the EasyTier Pool, the need to lower volume response times might dictate adding more storage to the SSD tier.



Key Capacity Metrics to focus on: Virtual Allocation %

Virtual Allocation (%)

The percentage of the physical capacity that is committed to the virtual capacity of the volumes in the pool. If the value

exceeds 100%, the physical capacity doesn't meet the demands for virtual capacity. The following formula is used to calculate this value:

[(Total Volume Capacity ÷ Pool Capacity) × 100]

This value is available for all pools.

Example: If the allocation percentage is 200% for a total storage pool size of 15 GiB, then the virtual capacity that is committed to the volumes in the pool is 30 GiB. This configuration means that twice as much space is committed than is physically contained in the pool. If the allocation percentage is 100% for the same pool, then the virtual capacity that is committed to the pool is 15 GiB. This configuration means that all the physical capacity of the pool is already allocated to volumes.

An allocation percentage that is higher than 100% is considered aggressive because insufficient physical capacity is available in the pool to satisfy the maximum allocation for all the thin-provisioned volumes in the pool. In such cases, you can use the value for Shortfall (%) to estimate how critical the shortage of space is for a pool. You can hover the mouse pointer over the percentage bar to view values for the total volume capacity and capacity. Total volume capacity total is the storage space on all of the volumes in the pool. For thin-provisioned volumes, this value includes virtual space. Capacity is the total amount of storage in the pool.

Use this metric to track capacity for A9000 Pools



Virtual Allocation % for Pools on an A9000

Since Physical Allocation % is not available on an A9000, we must use Virtual Allocation % to understand Pool utilization



Key Capacity Metrics to focus on: Zero Capacity

Zero Capacity:The date, based on space usage trends, when the pool runs out of available space.

Depleted is not a good conditionfor the Capacity Planner



Exporting Capacity Data for Trend Analysis



Trend Analysis with Excel

Exporting CSV data to Excel can help visualize growth, but can be

cumbersome and probably not practical on a large scale



Cognos Can Also be used for Historical Analysis

Cognos Main Menu

Pre-Defined

Reports

Storage Systems

Sub Category

Next is an example

of Storage Systems

Historical Capacity



Cognos Can Also be used for Historical Analysis

There are a number of pre-defined Cognos reports that show historical analysis, with fixed

reporting periods, up to 90 days. Unfortunately, there are no projections.



• /srm/REST/api/v1/StorageSystems/

Exporting Capacity Data via REST API

[

{

"Name": "XIV-214-1340004-214",

"id": "19379"

"Assigned Volume Space": "43",

"Available Pool Space": "25,465.90",

"Total Volume Capacity": "33,473.74",

"Unassigned Volume Space": "19,003.19",

"Used Pool Space": "13,862.81",

"Used Space": "11,546.79",

Another option for custom reporting is using the REST API. The capacity metrics within the

Spectrum Control Repository can be exported to create customer reports.



Spectrum Control Grouping Options

Groups allow you to visualize parts of the storage infra-structure to help

understand capacity and performance. In addition, you can specify Alerts that

are limited to the Group.

• Applications are really the main form of Groups. You create Filters to specify what is

included in the Application, although you can manually add an object if desired. An

Application Group can have sub-components.

• Departments are a way to group Applications to get a larger view of the storage

components. Like Applications, Departments can have sub-components.

• General Groups are different in the sense that you do not use Filters to automatically

populate the group. To populate a General Group, you add the storage objects

directly, either through selection or multi-select.

• Tiers are a form of Grouping where you can visualize your storage based on the type

of storage. Storage Tiers are generally grouped along the lines of flash storage,

spinning disks and SATA devices.



Application Groups Enable Visibility into Storage Consumption

View capacity consumption per

• Application

• Department

• Server

• Hypervisor



The “Database” Application Group with Related Resources

Keep in mind that unless the underlying infrastructure of the related resources are dedicated, then these resources represent shared capacity outside the group.



The “Database” Application Group Filter based on volume pattern

The filter brought in 15 member volumes



The “Database” Application Group Filter based on volume pattern

The Related Resources includes 34 volumes, when you add in all of the copy relationships



Creating applications

Ways to populate an application:• The servers / hypervisors running the application• Enter a volume name pattern that matches your volume naming convention• The storage pool that the application runs in• Manual selection of volumes



Email Reports Give A Good Overview of Application Consumption



The Department “IBM Apps” includes Databases plus more…

A total of 5 Applications



General Groups are Groups of Storage Objects

General Group objects can be

storage systems,

pools, volumes, disks, pretty much

anything.

The only caveat is

that if you want to

apply an Alert, it is

at the object level that you created

the group with.



General Group Alert at the Storage System Level – One Condition

If more granularity is needed, then you would create a group containing components of a storage device, like pools or nodes



General Group Alert at the Storage System Level – Custom

A custom Alert allows you to add conditions together, in this case the Alert is for Physical Allocation “and” Shortfall



Identifying Waste At The Storage System Level

Reclamation looks at inactive and orphaned LUNs to highlight potentially reclaimable storage



Identifying Waste Drilled Down To The Volume

By clicking on the Volumes hyperlink, we can see exactly which volumes are identified as being potentially reclaimable.



• Spectrum Virtualize enables some key capabilities for capacity planners that are not available to non-virtualized environments

• The key feature to exploit with Spectrum Virtualize is non-disruptive data movement

• Non-disruptive data movement means that free capacity can be moved from one place to another as need be without taking application servers down

• While you can not move active volumes non-disruptively between clusters, you can move Mdisks between clusters, once removed from a Pool

• In addition to moving capacity, data reduction techniques can be employed such as compression and thick to thin migrations at the Spectrum Virtualize layer to free up space

• If a FlashSystem A9000 is being virtualized, then capacity can be moved to storage Pools to take advantage of deduplication, compression and thin provisioning

• The bottom line is that virtualization gives capacity planners a number of non-disruptive tools that are not found in other environments

45

Spectrum Virtualize Family and Capacity Planning



• The Spectrum Accelerate Family of products include the FlashSystem A9000, the XIV

Storage System and Spectrum Accelerate software defined block storage.

• One of the key differences between this family of storage and other types of storage is

that the underlying architecture is grid based

• For the capacity planner, a grid based storage system means that Pools are more of a

logical construct rather than a physical construct

• Since Pools are a logical construct, they become more of a fencing mechanism to help

separate storage usage by application

• If the initial size of a Pool is not set correctly, then it is very easy to increase the size of

the Pool by changing the number

• For this family of storage, we use Virtual Allocation % to measure Pool utilization versus

Physical Allocation % on other devices.

46

Spectrum Accelerate Family and Capacity Planning



• Capacity Planners need to establish targets and understand their timelines

• Once key metrics are identified, initial focus will be on over all storage system capacity and utilization

• Secondary to the over all storage system, are the Pools within the storage system where capacity, utilization and growth can be identified at the application level

• To help visualize storage usage, grouping methodologies can be utilized to understand usage and growth over time

• Alerts should be utilized at the levels that make sense, either individually or by grouping applications or objects to be notified of changes in capacity and utilization

• Technologies like Spectrum Virtualize and Spectrum Accelerate can be exploited to give capacity planners more tools to manage capacity and utilization

47

Putting It All Together



Accelerate with IBM Storage Webinars

The Free IBM Storage Technical Webinar Series Continues in 2017...

Washington Systems Center – Storage experts cover a variety of technical topics.

Audience: Clients who have or are considering acquiring IBM Storage solutions. Business Partners and IBMers are also welcome.

How to sign up? Information, schedules, and replays:

To automatically receive announcements of upcoming Accelerate with IBM Storage webinars, Clients, Business Partners and IBMers are welcome to send an email request to

[email protected].

Located in the Accelerate with IBM Storage Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/accelerate/?lang=en

2017 Webinars:

Jan 19 - Spectrum Virtualize Updates Feb 2 - DS8000 Configuration & Monitoring using the GUI Mar 9 - Elastic Storage Servers & Spectrum Scale Software SolutionMar 16 - TS7700 Mgmt Classes & Copy PoliciesApr 27 - How to Develop a Workload Baseline to Evaluate Flash StorageMay 17 - DS8880 Storage Options for IBM iJune 21 – TS7700 BVIR What You Need to KnowJune 22 – FlashSystem A9000 UpdateJune 29 – DS8880 Thin Provisioning July 20 – Spectrum Virtualize EncryptionJuly 24 - Advanced uses of IBM Storage/RansomWare attacksAugust 10 - Accelerate with IBM Storage: Tape, What's NewSept. 7 - Utilizing the BVIR VEHSTATS Reports to Monitor Your TS7700 EnvironmentSept. 14 – DS8880 Technical Update (R8.3)October 3 – GDPS Update

October 10 – z14 and IO Infrastructure ModernizationNovember 1 - Spectrum Control Capacity Planning

November 16 - VersaStack - IBM’s On Premise Cloud Solution

Register Here:

https://ibm2.webex.com/ibm2/onstage/g.php?MTID=e3dad46fed77c9a7f

11cadc4173e2763a

November 30 - LTO8 Drive & related updates to IBM tape products

Register Here:

https://ibm2.webex.com/ibm2/onstage/g.php?MTID=e3b6902270711054

8491a3785652ce61e

December 5 - IBM FlashSystem A9000 & A9000R R12.2 Update

Register Here:

https://ibm2.webex.com/ibm2/onstage/g.php?MTID=e0e88c21e4431592

bb9887d9df8040d80

December 12 - DS8000 DSGUI Live demo

Register Here:

https://ibm2.webex.com/ibm2/onstage/g.php?MTID=e9cdd72f0e08f40f72

0efefd4699397e9

Spectrum Control Capacity Planning - IBM

Documents

Transcript of Spectrum Control Capacity Planning - IBM