How Startups can Prepare for Commercial Growth through Scale Testing
What Product Elements Can You Scale Test?
|ELEMENTS||SCALE TESTING CONSIDERATIONS|
|IoT Devices and Sensors||How many devices can your IoT product handle? Can it support additional sensors or device hubs?|
|Cybersecurity||How much traffic can your product handle without compromising user or client security?|
|Incoming Data||Can your product handle a larger volume of data efficiently, both operationally and cost-wise?|
|Analysis and Artificial Intelligence||Does scaling the amount of data collection compromise the accuracy of your product’s analysis and conclusions?|
|End Users||How many users can access your product without a degradation of services?|
|Request Rate||Does scaling up the number of user requests in a specific time frame impact your product’s ability to process data in a timely manner?|
How Would You Scale Test for These Elements?
The scale testing process, required resources, and chosen metrics will depend on a company’s specific product and goals.
|What type of tech products can be scale tested?||On-premise, private and public cloud-based software can all be scale tested.|
Hardware can also be scale tested to see whether scaling devices affects product usability/efficiency.
|What kind of resources are used for scale testing? How much is needed?||Organizations generally require CPUs, GPUs, and/or fast storage to scale test. These resources may be provided as cloud resources or as bare metal servers.|
Additionally, traffic generators can be used to simulate user traffic or stress APIs to measure a product’s performance when the number of users or frequency of calls is increased.
One major challenge is understanding what resources may be stressed or what infrastructure is needed before testing. This leads to issues when it comes to estimating the cost of testing on the cloud and purchasing personal infrastructure.
One solution to this problem is system monitoring. Monitoring can provide real-time or historical data on what resources are being used. This data can be used to identify bottlenecks or forecast what resources may be required for larger-scale loads without the cost of scaling until the system breaks.
System monitoring is provided by CENGN on our private cloud and bare metal projects by default.
|What is your company’s specific goal behind scale testing?||Before testing begins, identify what metrics you are testing for and specific criteria for knowing whether your testing was successful.|
There are typically three types of metrics used to determine scalability:
1. Number of users (total or concurrent)
2. Number of devices
3. Requests per unit of time (per second, per minute, per hour, etc.)
By incrementally scaling these metrics, you can measure the effects on different product attributes, like response time, throughput, latency, load-bearing performance, screen transitions, network usage, memory usage, and frequency of errors. These are the type of resources and performance metrics monitored by system monitoring software.
The metrics you choose to use for your scale tests will depend on your goals.
Often, a company’s goals regarding scale testing are informed by time constraints and their projected growth in the market. CENGN generally recommends companies scale test using projected performance for the next 18-24 months.
|What process should you use to scale test?||Establish a structured process for testing your chosen metrics that will validate future success with larger-sized customers or growth in the number of customers. It’s important to scale test your product in a logical progression that matches realistic business growth.|
There are generally two ways to approach scale/performance testing:
1. Given a certain resource set, how many users/requests per second, etc. can we support?
2. With a target user/request scale, what resources are required?
Scale Test Your Tech Solution With CENGN
As a non-profit looking to supercharge Canada’s innovation economy, CENGN offers the infrastructure and expertise needed for small and medium sized businesses to scale test their tech products.
These services are provided through a no-cost CENGN project to Canadian companies looking for commercial growth.
Most tech companies are experts in their specific product, not scale testing. CENGN leverages the experience gained from 200+ commercialization projects to help businesses determine their testing goals, identify appropriate metrics and required equipment, design a unique testing process, and scale test and validate their product.
Check out past CENGN Projects
Browse to learn more about the companies that CENGN has worked with, their innovative solutions, and how a CENGN project fit on their path for commercial growth.
October 27, 2022
Management Decision-Making AI With Advanced Symbolics
Advanced Symbolics is the first company to offer a probabilistic sampling of social media. So, askpolly, an AI developed at the University of Ottawa, can combine the scientific rigour of traditional polling research with the immediacy of social media listening.Read More Artificial Intelligence
October 27, 2022
Hands-On and Realistic Cybersecurity Training with Cyber Range
Field Effect’s Cyber Range allows instructors to design environments quickly, using drag-and-drop editing while replicating the environment for each student. Students are isolated from one another, need only a browser, and have the benefit of learning at their own pace.Read More Security
October 17, 2022
Global Command and Control
NORTAC Defence® is a leading provider of hardware and software for Situational Awareness and Command and Control at the tactical level. The platform provides organizations with secure web-based applications or in-premise solutions behind client firewalls.Read More Data Centre and Cloud