Introducing Propellers: Easily select the optimal cost and query speed for each Customer-Facing Analytics use case

Today, we are thrilled to introduce Propellers, an easy way for product development teams to select the optimal cost and query speed for their customer-facing analytics use cases.

In some use cases, such as analytics on a core product workflow, you might want to prioritize speed over cost. In others, like an internal app, you might want to prioritize cost over speed.

Propel’s architecture separates storage and compute, allowing each to be consumed, scaled, and priced independently. This enables you to select the optimal levels of query speed and cost for your use case.

What are Propellers?

Propellers are the unit of compute in Propel. The larger the Propeller, the faster the queries and the higher the cost. Every Propel Application (and therefore every set of API credentials) has a Propeller that determines the speed and cost of queries.

You can think of a Propeller like a car engine. The size of the engine (Propeller) determines the speed at which the car can go. But just as importantly, it also affects how much fuel (costs) the car uses.

A small Propeller can be thought of as a "compact" car engine. It's great for getting around town, and it's very fuel-efficient. But if you need to go on the highway, or if you have a lot of people or cargo, you're going to want a bigger engine.

Propellers determine the maximum number of records a query can process per second. Queries that are very complex or that scan a larger time range will require the processing of more records.

The table below describes the different Propellers, their capacity, and their price. The Propeller’s price is determined by the maximum number of records it can process per second.

How do Propellers work?

In order to query data from Propel, you first create a Propel Application. Your Propel Application includes settings, like OAuth 2.0 credentials, as well as the Propeller to use when executing queries.

You can choose a Propeller ranging from P1_X_SMALL through P1_X_LARGE when creating your Application, and you can resize it at any time.

Next, when you query data from Propel, you include your Propel Application’s access token (see authentication). Propel checks your access token, identifies your Application, and executes your query with your Application’s configured Propeller.

How do I choose a Propeller size?

If you're not sure what size Propeller you need, don't worry! The Propel Console can help you select the optimal Propeller size for your use case. Here’s how:

1. Go to the Metric Playground in Console.
2. Create a representative query for your Metric.
3. This should be a query your Application will actually execute.
4. Starting with the smallest Propeller, run the query and check the query statistics.
5. If the query was too slow, choose the next highest Propeller from the drop-down.
6. Repeat until you get the desired cost-to-performance.

Here are a few rules of thumb to follow when choosing Propellers:

• Start small and move to bigger Propellers as your Data Pool grows.
• Choose bigger Propellers when speed is important or your Data Pool is large.
• Don’t automatically choose the biggest Propeller just because you have billions of records. Filtering by time range and tenant ID significantly cuts down on the number of records, meaning even medium-sized Propellers can perform well on larger Data Pools.

What are the use cases for Propellers?

Propel's customers use Propellers for many different things, including the following:

Optimizing for fast response times on large datasets

Imagine the main dashboard of a SaaS product that has to query a ton of data and is core to every workflow in the product. Customers have no tolerance for slow products, so you'll want that dashboard to load blazingly fast. A large Propeller lets you optimize for speed here in the customer-facing product.

Optimizing for cost on small datasets and non-core workflows

Not all customer-facing workflows handle massive amounts of data or need sub-second response times. For these non-core workflows that query small datasets, you could use a smaller Propeller and avoid paying for performance that isn’t needed.

Similarly, employees might be OK waiting 5‒10 seconds for a query to load in an internal tool. Here, you might want to keep costs lower and go with a smaller Propeller.

Optimizing for development and production environments

Production environments that serve customers might want to use large Propellers, while development and staging environments can get away with smaller Propellers. Again, Propellers enable you to pay for performance where you need it.

<aside><div>💡</div> <div style="padding-left: 40px; margin-top: -24px; position: relative; padding-bottom: 0;">Not all use cases are created equal, and that's where Propellers come in. With Propellers, you can easily select the optimal cost and query speed for each customer-facing analytics use case.</div>

</aside>

How do I get started with Propellers?

You can use the Propel Console to create an Application and select the size of Propeller. You can learn more about Propellers by reading the documentation.

If you don't have your Propel account yet Join our waitlist!

We are onboarding users first as fast as we can. We can’t wait to see what you can build with Propel!

Follow us on Twitter @propeldatacloud or subscribe to our email newsletter.

‍