Threading on ticker mode

[zsmoore]

Hello all,

We have been using ticker mode for awhile now and are moving away from it due to a troubling amount of threads spawned / cpu / gc overload.

We currently have 47 DLs in production using a dispatch all basic predicate and looking at the code

java-dataloader/src/main/java/org/dataloader/registries/ScheduledDataLoaderRegistry.java

Line 237 in 918203c

if (!closed) {

See that dispatches are scheduled on a per DL level rather than a dispatch all, if i am doing calculations correctly then I believe a tick time of 5 ms , with 47 DLs, and a call time of 400ms == 3760 threads per call.

There is room for tuning here but I was wondering why we need a per dataloader dispatch vs, a single callback which runs a dispatchAll on a single thread for the reschedule. This seems like it would significantly decrease thread overhead for the ticker mode.

Beyond this we are moving towards a more deterministic, level by level deterministic dispatch which i have been thinking about how to contribute back to OS but haven't found a great way to generalize it yet.

Wondering about thoughts specifically on if a per DL reschedule is necessary vs a dispatch all at once when the default dispatch all predicate is used.

Thanks!

[andimarek]

Hi @zsmoore,

Can you help clarify a few things?

What is ticker mode exactly?

In GraphQL Java we always had a per level mode by default, do you mean that when you talk about more predictable mode?

We just recently added a new mode in GraphQL Java which is called exhausted strategy, which works similar to the JS dispatching mode.

Best

[zsmoore]

Sure thing, we are using dataloaders without graphql in our app to benefit from the caching per request + batching decentralization.

Ticker mode I am referring to is with the scheduledDLRegistry here -
https://github.com/graphql-java/java-dataloader?tab=readme-ov-file#scheduleddataloaderregistry-ticker-mode

So unfortunately we are not able to benefit from the GQL general functionality as we are only leveraging the underlying library.

[zsmoore]

I will definitely look into this tomorrow, thank you for the thoughtful response and i'll let you know how it goes! Once i was going over this again i realize i might not even need the fallback scheduler with the futures and might be able to just take the idea of being busy as you mentioned as a signal that the request is not finished

Outside of this im wondering what the long term plan is for ticker mode in scheduled data registry. I think it makes sense for small projects but most likely not for large scaling and wondering how you all think about the experimental api guidelines for the project

[andimarek]

We will probably have to tighten a bit the guidelines around ticker mode, I agree. The thing is that so far we don't have a better data loader native solution for chained data loaders, so if you are in a pure data loader world, next tick is probably your best bet.

cc @bbakerman for his thoughts on this matter, as he is the main author of dataloader.

[andimarek]

@zsmoore do you have any update? I would be happy to look more into it PR #243 if you think that is valuable for you.

[zsmoore]

Working on this this week , hopefully should have an update for you soon. We ended up fully ramping the deterministic strategy to 100% and are seeing completely stable servers now

[zsmoore]

@andimarek I went ahead and put this together as a separate PR from my fork here - #261