On-Demand Workshop Recording

On-Demand Market Data Access from KDB-X

In this workshop, Peter Simpson demonstrates how you can use on-demand market data, without having to self-collect, store, and manage it. The workshop will focus on retrieving both historic data and trading analytics metrics into KDB-X.

This session is designed for KDB financial professionals interested in simplifying their market data pipelines for quantitative development within KDB-X, and accelerating data onboarding from months to minutes.

Introduction to OneTick Cloud:

OneTick Cloud is a high-quality, on-demand managed time-series data and analytics platform that provides instant, global, AI-ready market data, seamlessly fueling compute, research, and analytics engines.

Hydrate KDB-X with OneTick Cloud:

Data hydration is the process of taking raw, fragmented, or asynchronous financial data and standardizing, enriching, and cleaning it so it becomes AI-ready. It bridges the gap between raw unstructured data (like raw exchange ticks) and actionable machine learning and quantitative analysis.

The hydration process involves several steps:

  • Cleaning and Normalization: Eliminating errors, correcting biases, and aligning irregular timestamps across hundreds of different trading venues.

  • Temporal Fidelity: Structuring the data so it is point-in-time correct, which prevents "look-ahead bias" during backtesting and agentic reasoning.

  • Enrichment: Filling critical data gaps and adding contextual depth, such as cross-referencing prices with market regime scores or news feeds.

For algorithmic trading and AI agents, hydrated data is crucial because it eliminates the "data tax"—the 70% to 80% of time quants typically spend cleaning feeds. By providing fully harmonized, audit-ready time-series data, it allows trading systems to operate without the data gaps that cause hallucinations or inaccurate predictions.

Benefits of the Managed Data Service:

OneTick Cloud offers a single vendor end-to-end solution, significantly reducing the time to value. KX and OneTick are the only vendor delivering AI-ready, hydrated, temporal market data as a managed service. Our data is pre-normalized across 250+ venues and 30+ years of history, point-in-time with no look-ahead bias, machine-readable from day one, and fed natively into Python, SQL, and KDB-X.

Save time, money, and resources by letting the KX OneTick team clean feeds, map symbols, and align timestamps so your quants, analysts, and AI models can do their work.

Workshop Description:

In this workshop, Peter Simpson demonstrates how you can use on-demand market data, without having to self-collect, store, and manage it. The workshop will focus on retrieving both historic data and trading analytics metrics into KDB-X.

Topics Include:

OneTick Cloud Datasets:

  • L1 through to L3

  • Real Time, Intraday and Historic

  • Global Equities, Futures, Spreads & Options

OneTick Cloud Module:

  • Install

  • Instantiation & Configuration

Retrieving Data & Trading Analytics Metrics:

  • Trades, Quotes, NBBO, 1 Minute Trade & Quote Bars, and Daily Metrics

  • Book Depth from Market by Level (MBL) to Market by Order

  • Applying Corporate Actions to Adjust Historic Pricing

  • Querying for Futures Continuous Contracts

  • Querying for Alternative Symbologies including Bloomberg, FIGI, ISIN, SEDOL and CUSIP

  • Market Structure Metrics

  • Market Microstructure Metrics

  • TCA Metrics, including Period Stats & Mark Outs

This session is designed for KDB financial professionals interested in simplifying their market data pipelines for quantitative development within KDB-X. And how to accelerate data onboarding from months to minutes.

Watch the Recording:

 

Webinar Transcript:

Introduction


Hello. Thank you for joining our webinar today, part of a webinar series all about OneTick Cloud. This workshop today is focused on accessing on demand historical market data from KDB-X. My name is Hannah Paynter, and I will be your host for today's session.

Housekeeping as people join us:

  • Polls during the webinar, we'll hold a number of polls. A few of them are running now. We really appreciate your participation and it helps us to cater today's presentation and future workshops towards our audience.

  • Any questions, we have a Q&A session at the end of the demonstration. You can ask questions at any time using the Q&A button.

  • Technical questions, if you have any issues during the webinar whatsoever, please use the chat button to let me know.

  • A recording of the webinar will be made available as soon as possible later in the week. It'll be sent out to all who registered, it'll be posted on both the OneTick website and the KX website.

At this point, I will hand it over to Peter Simpson, our OneTick Product Owner.

Demonstration


To query OneTick historical market data from KDB-X, I fire up Q, open up a port, then I'm gonna use Versus Code and the KX extension.

Make sure I'm connected.

Yep. Connected.

It says I'm connected, so let's try and load the module.

Okay. My module's loaded. It's given some instructions on how to use it.

Now let's execute the current selection.

Here's my SQL.

Let's take our SQL and execute it.

And now I get my results.

The results are returning as a Q Table.

 

In this case, I'm just asking from the US equity sample data set, give me apple trades within this time range.

And the data is being returned with time stamps being in UTC.

 

So let's say I want time stamps being returned in New York time.

Well, I can specify the timestamp, then build a dictionary of my SQL and my timestamp, And I can execute that dictionary.

This time around, my timestamps come back as New York time. Says 4:00 AM because the early session opens at 4:00, and here are the first trades for Apple.

Now we're storing data with the exchange symbol, but I can query with different symbologies.

If I prefix the database with b sym, then I can specify a Bloomberg symbol.

You can see the Bloomberg symbol now returning data.

I can do the same for a FIGI.

In this case, I'm specifying the composite FIGI by just prefixing with FGC. Again, I get my data.

Same for a CDL.

Same also for an iSIN and for a CUSIP, and I get the idea. Just prefix the database with the selected symbology, enter the symbol, and get back the data.

 

Now I can pull back trades, quotes, NBBO, join the data together, aggregate filter.

If I want to pull back booked at, I use this OB snapshot function.

This case, I'm specifying from the table p r l four for HSBC.

I'm getting the data back.

One row is representing a level in the book and a side of the book.

I can also filter the book by specifying the maximum number of levels.

So I'm going just down to five levels and specify I want the book output every sixty seconds, for example, or at a particular point in time or continuously.

So now we see the data carrying on.

And in this case, we can also change the format.

So if I wanted the book with one row representing a level in the book with a bid and ask, I can go to this format.

Or if I wanted one row representing a book update with lots and lots of columns, I can go to this format where I have columns for bids and asks to the number of levels I specify.

 

Now rather than querying equities, like a query futures or options for that match or spreads. In this case, I'm specifying the exchange product code, expiry month, and expiry year. I'm getting my data back for the December 2024 Brent crude contract.

I'm gonna specify my end of day records. I've got my open interest, OHLC, settlement price as well.

Now rather than just retrieving a particular contract, we can retrieve a continuous contract. So I'm specifying the front month.

This time, I get that back.

Or I could specify the continuous contract with a Bloomberg symbology. And this data comes back.

Rather than retrieving historic data, I could retrieve intraday data, or even the latest data from last value caches.

So for the US equities, I use the snap table for the last value cash, which combines both trades and MBBO.

From that, I'm getting back every price, open, close, etcetera.

Let's just change that to give me back every column.

Execute again.

Now I'm getting data back into KDB again.

So in this case, I issue the queries from Q, just writing a SQL statement that goes to the OneTick Cloud servers, executes, either putting back real time data, intraday data, historic data, comes back with the results. The results end up as a Q Table.

And then with those Q Tables, I can count how many rows have been returned.

And you can see it's nearly 14,000 instruments in the US equity table.

 

Q&A Session


Alright. Thank you, Peter. That was a great short and concise presentation on how to access that market data. We have a few questions coming in. Thank you to all who have submitted questions and participated in those polls.

Let's see. So first question is, which symbols or symbologies can I query against?

Okay. We saw a few, when I was walking through.

So we are storing data with the exchange symbol, but you can also query, with the Bloomberg symbol, the FIGI, the iSIN, the CDL, or the CUSIP.

If you have a CDL or CUSIP license, then you can query with one symbology and pull back what the matching CDL or CUSIP is.

 

Great. Another question here. Can I query for real time data?

Yes.

From an access point of view, it's exactly the same as querying for historical data who are just issuing the SQL statements, either for giving me back intraday data for the current time, or you're saying give me back the last price across every instrument within my selected venue.

The only real difference is around exchange permissioning. For you to be able to see a real time venue, your account needs to be permissioned to see that venue in real time, and we need the exchange agreement signed for you to do that.

 

Great. We have one audience member here who uses kdb+ most often as their data analytics method.

So the question from them is, can I use kdb+ instead of KDB-X?

Not currently. The module that we've created is KDB-X specific. But if you can provide us, your details, we can go away and talk internally.

So at the moment, KDB-X provides an easy way to add modules. And in the case of this module, we're going out to OneTick Cloud, issuing the SQL query, getting back the results, sending that back via arrow, so in a binary format, into KDB-X, and then it's being converted from arrow back into a Q Table.

So if you provide us your details, we'll get back to you on how we could expand this functionality from KDB-X to kdb+.

 

Great. Let's see. A next question here. Can I query for book depth?

Yes. We typically retrieve the PCAP files from each venue.

That data gets loaded, both for kind of level one, so trades and quotes and book depth. You can either simply query every underlying kind of order update message, or you can use our OB snapshot or OB summary functions to query the book down to a particular filtered view, which might be by level, by number of shares, by trade value, by spread, by price skew, and that book can be pulled back by book update by book update or periodically as kind of book bars. You're choosing how you want to retrieve the book, and there are, as we saw in the little demo, three different representations of the book.

Or you can use our kind of summary function where you're asking for more analytical questions. So if I wanted to trade a certain size or a certain value, how deep in the book would I need to go, and what's the VWAP bid and ask and the VWAP spread to be able to achieve that trade size.

It's exactly the same process. Issue the SQL, get back the results, and not have to worry about collecting and storing and managing the book data yourself.

 

Great. Kind of a follow-up question to that. How could you learn more about the OneTick SQL syntax?

Okay. Probably two ways. First way is you just go to Google, type OneTick SQL documentation, and you'll come into our public, Sphinx based documentation.

And that has an AI assistant built in. So you can either read through the documentation or just ask a question, and that will come up with the syntax for you.

Or you can, use our MCP server, connect up, and then ask through your IDE of choice, let's say, Versus Code, or just kind of Claude code, ask the natural language question, and that will that will come back with what SQL you should use to execute.

 

Great. Let's see. Next question here. How long does this take to get set up?

Okay. The installation of the module probably takes ten to thirty seconds by the time you've downloaded it and copied it into your KDB-X install.

That comes with prebuilt credentials. So from start to finish, in terms of accessing the samples, that takes thirty seconds. In terms of accessing more datasets, which could be any market globally, whether that's equities, futures, or options, Once the paperwork is signed, we set you up, which is just giving you your own credentials, and then you have access to the full history.

So once paperwork's signed, that's probably an hour or so before you can get your new credentials.

It takes a bit longer in terms of real time because then we also have exchange agreements to be signed.

But it's literally hours. There is no backloading of data, waiting for data to be loaded onto your local storage, providing servers. It's just, “here's the data.” You have pretty much immediate access to it.

 

Fantastic. Thank you. And all that market data coverage is updated on the OneTick website as well. Let's see. So we have another question here. How can I query across fragmented liquidity?

Okay. So especially with equities in various markets around the world, there are multiple venues.

In the case of the US, we have probably twenty different kinds of let venues, and there is the US SIP, so a consolidated tape that's provided, which combines all those venues together, and we provide that in our US comp dataset.

Outside of the US, there is no consolidated tape, so we generate them for you. And we do that for various markets around the world. For certain markets like Australia, India, Japan, there are a few markets.

The same goes for Korea, Taiwan and Mexico. When we're looking at Canada, there are ten to fifteen markets. When we're looking at Europe, we're combining around twenty six different countries and a hundred well, a hundred and fifty plus different mix.

So in these cases where there's not a consolidated tape, we're creating one for you. We're creating a our own consolidated BBO. So there's a Canadian BBO. There's a European BBO, Japan, Australia, etcetera.

You can then query this consolidated view of the data and see one place to access every single trade for that particular region and benchmark that trade against the NVBO that we're generating for you. Or you can use our kind of analytical features to create your own NBBO based on the venues that you trade if you want a subset.

 

Great. We have a question here that asks: Can I query for RICs?

Okay. That symbology we do not support because it is specific to Refinitiv or LSEG, so we can't reuse that. But if you want to instead use the Bloomberg symbol, FIGI, ICE, and c dot CUSIP, or just the exchange code, then we can query with that.

 

Great. Let's see. Couple questions about specific events. How do I know that a corporate action has occurred?

Okay. Now we are storing the data as it's received. We're applying trade corrections or trade cancellations, but it is the data as we receive it.

We also store corporate action adjustment factors and the type of corporate action that's occurred. And we provide functions so that you can query the data unadjusted or query the data after corporate actions have been applied.

And it could be that you just want to query when splits occur or you want to also include dividends. You're choosing how you want to adjust the data based on the corporate actions that you're interested in.

And that's just one additional function into your SQL statements to retrieve the data. And then you could retrieve adjusted or unadjusted and adjusted as additional fields.

Or you could choose just to retrieve the unadjusted and retrieve the adjustment factors and do your adjustment locally. But we're making it so easy. You just pull back the adjusted data.

 

And then, kind of a follow-up. Is it similar for continuous contracts rolling?

Yes. So we're storing all of the futures data contract by contract with its history going back.

There may be times where you don't want to look at individual contracts. You want to look at the continuous contract.

For the first twelve months or the first twelve front months, we provide access to continuous contracts based on expiry. So when the contract expires, we'll roll onto the next.

For the front month contract, we also provide alternatives where we pick the contract with the maximum volume or the maximum open interest. So you have a few different ways to query.

When you're querying with a continuous contract, you're using either the exchange code and then a slash and the number. So doing it in the front month or the second month, etcetera.

Or you could also query with the Bloomberg code for the continuous contract as well, either looking at the contracts based on expiry or contracts based on maximum trading volume.

 

K. We have a question that was actually submitted during registration. Any tier for community use just for exploring data? A free trial?

Yep. So as part of installing the module, you have access to a subset of the sample data, which effectively covers the first week for tick data for various markets during twenty twenty four or the first week of 2024. So that covers US equities, European equities, Canadian equities, Australia, Japan, and also LSE where we also provide book depth.

Then, also CME and ICE and EUREX, we also provide samples.

Now you can upgrade from kind of the standard module, to be provided credentials where rather than looking at just the first week, you look at the first three months of twenty twenty four, or and that also includes access to common routes for Opera where you're looking at some US stock options.

If you wanted to go further, then it's a paid subscription where you're signing up to accessing the full datasets across the history that you require.

And you may be doing that where you want the last twenty years of data or you may be only looking for the last, let's say, rolling year of data going back for either the markets you want to subscribe for or a particular set of symbols across markets.

So start off with a limited set of sample data across both equities, futures, and options.

Then that expands to three months of data.

Again, samples during twenty twenty four or for US options, first month of twenty twenty five. And from there, you can go to a paid subscription where you look at all of the datasets that we provide.

 

Great. That leads me to this next submitted question, which obviously is more individual. How is this priced?

Yep. So there are a few different aspects here.

Are you interested in accessing a whole venue? So do you want every single US equity, for example, or European equity? Or are you more interested in just, let's say, the top six hundred equities or the top one hundred futures products, you tell us which you want. We'll provide the most effective option. So do you want access to the whole venue, or do you want to be more so venue priced or symbol priced. And then how much history do you want to receive, ranging from one year of history going back to to twenty years.

And, also, do you require access to real time data, or are you just looking at t plus one?

And then how much compute do you require to run your queries? Are you just dumping data down, or do you want to run the SQL to aggregate the data and just pull back the summary data?

So we'll work with you based on your requirements. Our existing customers typically start subscribing to a small range of data, and over time, they get more comfortable and expand their coverage universe, either adding new markets or increasing the history that they subscribe to.

 

Great. Thank you, Peter. Next question here we have, how often does coverage change?

Typically on a monthly basis, where we're adding new venues, that is less now typically around new venues and more around adding book depth for an existing venue. There are also regulatory changes.

So for example, for the US SIP, we've added additional tables for OddLock quotes and Odd Lot quotes in part of the MBBO that happened in June.

We're also continuously adding new AI feature sets. So last month, we added for the US enriched trade table.

So rather than just putting a table that includes every trade historically, this table includes every trade and additionally, the prevailing quote from the NBBO and also markouts based on the NBBO and around thirty markouts ranging from minutes before two minutes after the NBBO.

Just to make your analytical questions easier, rather than having to do a whole load of compute yourself, you can just go to the AI feature set and pull back the dataset that's enriched for your purpose.

 

I think we might have already touched on this one, but we just had a question come in.

How can I see the available databases, schemas, and field values?

Okay. If you go to the OneTick website and click on market data and coverage, you can see all of the datasets that we provide. That's also linked from the KX website. And there are details to drill in and then see every single schema that we provide. So under underlying each dataset will have a series of databases, which will have a series of tables, either the kind of underlying trades quotes, MBBO, book depth, but also derived data sets like one minute trade in quote bars, and also day summaries where we roll up and kind of which you'll see in split volume by trade types, and then also add in the kind of feature sets such as kind of enriched trades. So it's easy to go to the website and just look.

Also, all this information is available on GitHub. If you go to the one market data GitHub, you can pull back the information there. And also through our MCP server, you can ask a natural language question, like, which database holds Italian equities and come back with the result, which will be Milan, but also there are other datasets, kind of pan European datasets that also hold Italian equities. Then we have also our European composite, which aggregates all of that flow across each venue.

So lots of ways either using our MCP server or using our websites or GitHub.

 

Perfect. Let's see. One of our last questions here before we wrap up, can you just get access to the raw data and not use the API?

Okay. Yes. If you're accessing the raw data, then we provide an s three bucket where all of the data is available as Parquet.

And it's partition Parquet, partition by date based on each table. So the hierarchy is type of data, let's say equities, then the venue, then drilling down to the table of data, then down to the date and the Parquet file. When we provide access to Parquet, you need to be permissioned to see that whole venue.

If you want just to have access on a per symbol basis, then you need to be going by their API.

 

Right. Okay. Last question. Pretty simple. Can you query with QSQL?

Okay. Not currently. You're using the kind of OneTick SQL syntax to pull back data.

We're looking at how we can provide queue access later on to the year in the year in our kind of OneTick Cloud environment. So for the moment, from a Q client, you're using the module to issue OneTick SQL and pull back the data into a Q table.

 

Okay. Fantastic. Peter, any last thoughts about on demand market data from OneTake Cloud?

Please try it. It is, as we saw from the demo, it's a thirty second install, issue the SQL query, and come back with the data. The documentation's on the KX website. Ask us if you have any questions.

 

Okay. Fantastic. Thank you to everyone who registered and submitted questions, participating in those polls. We'll follow-up with some more proprietary questions privately following the event.

If you have any colleagues who might be interested in this topic or watching the quick, I think the demo was about seven minutes, please do share this recording. We're happy to answer any questions via email. Peter, thank you so much for your time today. To learn more about OneTick, please reach out to us at onetick.com or kx.com or via email, info@onetick.com.

Thank you.

Thank you.