Scale Testing

What Is Scale Testing?

Scale testing is used to identify the performance of a system or network when the number of users, devices, data points or transactions on a product increases. The actual tests performed depend on the type of product and a company’s goals. Scale testing makes it possible for technology companies to determine if their platform currently operates at scale. If not, scale testing may also show the resources needed based on future estimated workloads in the market.

Scale Testing Helps Companies Plan Ahead

Scale testing measures a product’s ability to meet the growing needs of its user base and support projected growth. It may also allow tech companies to determine the costs and resources of growing (like an increase in storage or compute on the cloud), as well as any reconfiguration that needs to take place to mitigate slower response times or errors resulting from scaling up their number of clients or size of deployments.

Scale testing also provides companies with proof of their ability to scale, which helps them commercialize their products by providing valuable reference for customers and investors.

What Product Elements Can You Scale Test?

ELEMENTSSCALE TESTING CONSIDERATIONS
IoT Devices and SensorsHow many devices can your IoT product handle? Can it support additional sensors or device hubs?
CybersecurityHow much traffic can your product handle without compromising user or client security?
Incoming DataCan your product handle a larger volume of data efficiently, both operationally and cost-wise?
Analysis and Artificial IntelligenceDoes scaling the amount of data collection compromise the accuracy of your product’s analysis and conclusions?
End UsersHow many users can access your product without a degradation of services?
Request RateDoes scaling up the number of user requests in a specific time frame impact your product’s ability to process data in a timely manner?

How Would You Scale Test for These Elements?

The scale testing process, required resources, and chosen metrics will depend on a company’s specific product and goals.

QuestionsAnswers
What type of tech products can be scale tested?On-premise, private and public cloud-based software can all be scale tested.
 
Hardware can also be scale tested to see whether scaling devices affects product usability/efficiency.
What kind of resources are used for scale testing? How much is needed?Organizations generally require CPUs, GPUs, and/or fast storage to scale test. These resources may be provided as cloud resources or as bare metal servers.
 
Additionally, traffic generators can be used to simulate user traffic or stress APIs to measure a product’s performance when the number of users or frequency of calls is increased.
 
One major challenge is understanding what resources may be stressed or what infrastructure is needed before testing. This leads to issues when it comes to estimating the cost of testing on the cloud and purchasing personal infrastructure.
 
One solution to this problem is system monitoring. Monitoring can provide real-time or historical data on what resources are being used. This data can be used to identify bottlenecks or forecast what resources may be required for larger-scale loads without the cost of scaling until the system breaks.
 
System monitoring is provided by CENGN on our private cloud and bare metal projects by default.
What is your company’s specific goal behind scale testing?Before testing begins, identify what metrics you are testing for and specific criteria for knowing whether your testing was successful.
 
There are typically three types of metrics used to determine scalability:
 
1.     Number of users (total or concurrent)
2.     Number of devices
3.     Requests per unit of time (per second, per minute, per hour, etc.)
 
By incrementally scaling these metrics, you can measure the effects on different product attributes, like response time, throughput, latency, load-bearing performance, screen transitions, network usage, memory usage, and frequency of errors. These are the type of resources and performance metrics monitored by system monitoring software.
 
The metrics you choose to use for your scale tests will depend on your goals.
 
Often, a company’s goals regarding scale testing are informed by time constraints and their projected growth in the market. CENGN generally recommends companies scale test using projected performance for the next 18-24 months.
What process should you use to scale test?Establish a structured process for testing your chosen metrics that will validate future success with larger-sized customers or growth in the number of customers. It’s important to scale test your product in a logical progression that matches realistic business growth.

There are generally two ways to approach scale/performance testing:
 
1.     Given a certain resource set, how many users/requests per second, etc. can we support?
2.     With a target user/request scale, what resources are required?

Scale Test Your Tech Solution With CENGN

As a non-profit looking to supercharge Canada’s innovation economy, CENGN offers the infrastructure and expertise needed for small and medium sized businesses to scale test their tech products.

These services are provided through a no-cost CENGN project to Canadian companies looking for commercial growth.

Most tech companies are experts in their specific product, not scale testing. CENGN leverages the experience gained from 200+ commercialization projects to help businesses determine their testing goals, identify appropriate metrics and required equipment, design a unique testing process, and scale test and validate their product.

Check out past CENGN Projects

Browse to learn more about the companies that CENGN has worked with, their innovative solutions, and how a CENGN project fit on their path for commercial growth.

July 15, 2022

RockMass

Revolutionizing Access to Geospatial and Geological Data in Underground Mines

RockMass is a mining and geosciences technology startup focused on data capturing and mapping for technical services. Their technology improves safety in underground environments, labour efficiency, optimized analysis for ground support design, block modeling and mine planning.

Read More  Internet of Things Smart Mining

July 14, 2022

Vivid Machines

Revolutionizing Fruit Production with Vivid X-Vision

The Vivid X-Vision system captures plant-level data to see plants’ chemical and physical properties, help with early-stage issue detection and automated counting and sizing of fruitlets through yield prediction.

Read More  Artificial Intelligence Smart Agriculture

July 12, 2022

Mien Tzu

Full-Experience Event Hosting on a Virtual Face-to-Face Platform

Mien Tzu is a revolutionary communication platform that provides full-service event support and is designed to help engage face-to-face networking in the virtual world.

Read More  Data Centre and Cloud