schelcc | blog on schelcc

C++ Thread Safety, Reinventing The Wheel, and Making My Life Easier

foo Wed, 23 Sep 2026 18:33:53 -0400

Alternative title: Carcinization? In my C++ codebase? It’s more likely than you think.

Whilst the job search continues, I thought I’d get out a quicker writeup about a lifesaver of a tool I’ve recently implemented in my ongoing project indy-tui¹ and how I ~~stumbled~~ iterated my way to it.

Background

By its nature, indy-tui requires keeping several threads up and running – when managing an interface, user input, and a live websocket session, one’s hand is slightly forced. Likewise, some amount of shared data across threads is largely unavoidable. Hence, it’s time for locking.

This project has been my first “real” go at an actually multithreaded application; a desire to gain some experience with multithreading, both generally and with C++’s Standard Library, was a strong factor in my motivation to take it on.² As such, the git history demonstrates me walking across the learning curve – first a lot of std::mutex and std::scoped_lock, followed by my own ill-fated RAII wrapper to add logging to lock acquisition³, a handful of std::atomic misunderstandings, only then finally landing on std::shared_mutex.

Thankfully, almost every instance of a need for shared data in this project⁴ is a “one writer, many readers” situation, so std::shared_mutex is really the best option. Until recently, I was satisfied giving each of my classes a shared mutex (or multiple to give load bearing components their own) and keeping access to them and their related data tightly behind getters and setters. This was largely fine, it didn’t feel the best but it seemed simple.

It seemed simple.

I like dataclasses

For better or worse, I quite like dataclasses. I like more semantic code what can I say.

For a UI rework inching towards some adaptation⁵ of MVC, I needed to pull out the UI logic I had put inside the leaderboard. That work was in there to begin with because the leaderboard needed to access the vector of drivers, as well as a collection of event information (track name, laps to go, current flag, etc.), and with these being behind several shared mutexes the UI logic needed the ability to get locks. Why have one responsibility when you could have more?

I like dataclasses, so the obvious solution to me was to pull the event information out into some EventInfo struct, with fields for each entry, and then have the leaderboard hold around an instance of it:

struct EventInfo {
 // We aren't guaranteed to get the data for these fields all at the
 // same time, so we populate them as we get them, hence the
 // optionals
 std::optional<std::string> event_name;
 std::optional<std::string> track_name;
 std::optional<int> laps_to_go;
 // ...
};

Fine enough, but this is shared data – gotta have those mutexes. But, what gets a mutex? And who is responsible for it? Some of these fields are updated much more frequently than others, and most of them are read each render loop; if the full struct gets one mutex, we could end up with a lot of contention. Conversely, if we want to give each field its own, that’s both a lot of annoying boilerplate and making the usage more annoying as I’d have to litter the codebase with mindfully-scoped lock wrappers (or do manual locking and risk forgetting an unlock) – this would look something like:

struct EventInfo {
 std::shared_mutex event_name_mtx;
 std::optional<std::string> event_name;

 std::shared_mutex track_name_mtx;
 std::optional<std::string> track_name;

 // ...
};

/* ... */
// Use somewhere in the codebase, say to check if we have the event name yet

// Open a tighter scope so that the lock doesn't live longer than it needs to
{
 std::shared_lock event_name_lock(event_info.event_name_mtx);
 if (event_name.has_value()) {
 // ...
 // event_name_lock is alive for all of whatever's in here
 } else {
 // ...
 // and all of whatever's in here too
 }
}

That’s not very fun. It’s even less fun when you need multiple locks at the same time.

This was the point where I had justified a tangent of trying to find a remotely ergonomic design. In an ideal world, I wanted a type (or macro)⁶ which could wrap any type and:

Add a shared mutex
Handle getting and setting in a more ergonmic way than manually acquiring std::shared_lock and/or std::unique_lock
If at all possible, do (2) in such a way that trying to modify the underlying object when taking a shared lock becomes a compilation error, as well as attempting to get a unique lock on a const reference to the wrapped object
When necessary, I can still hold a lock on the object for an arbitrary amount of time (e.g. need to work on several objects and need all locked to do so)

Surely⁷, the shared mutex and manual use of the shared/unique locks is perfectly viable. I just don’t like it. Also it’s my learning project and I’ll overengineer types if I want to.

With that, let the wheel reinvention commence.

The triangular wheel

First is the solution that truly missed the mark.

For whatever reason I was quite stuck in OOP brain when working on this⁸, so immediately what came to mind was a wrapper which simply defined a getter and setter, taking a shared lock and unique lock respectively:

// Omitting constructors and whatnot
template <typename T>
struct LockWrapper {
 T get() const {
 std::shared_lock lock(mtx);
 return obj;
 }

 void set(T t) {
 std::unique_lock lock(mtx);
 // In reality you'd want to take the input as a forwarding
 // reference and std::forward it here but there are more pressing
 // matters
 obj = t;
 }

private:
 T obj;

 // By qualifying this as mutable we can take a shared_lock in a
 // const-qualified method
 mutable std::shared_mutex mtx;
};

A couple problems here.

A glaring issue is the signature of get() – we incur a copy every single time we call get()! This is quite bad. Even worse, if T is not copyable it’s fully incompatible with this wrapper.

Another issue is what happens when you want to first assign to the wrapped value and then use it? You might say to return a reference to the wrapper in set:

LockWrapper& set(T t) {
 std::unique_lock lock(mtx);
 obj = t;
 return *this;
}

And then you might use the two methods together:

// Let some_field be a LockWrapper of some default constructible and
// assignable type T
some_field.set(T()).get();

This “works”, but there is a suspiciously elephant-shaped object in the corner that is begging to be acknowledged: we release the unique_lock and then some time later take the shared_lock. In other words, we are not guaranteed that some_field has not changed again between set() and get().⁹

Regardless of if I’m wrong and there is somehow a guarantee that the two calls will always directly follow each other, I still threw this solution out – the forced copying alone is a nonstarter, and this doesn’t allow me to actually hold onto the lock for longer if needed.

The hexagonal wheel

After the above failure, another component of the “right” solution became clear: the lock itself has to be handed off alongside the data. To take advantage of the whole RAII wrapper thing, the lock needed to be given to the caller and stored in some way. That is, the lock must be an lvalue at the callsite. Also, we should be sure to not copy literally everything – we should be returning references to the wrapped object.

Fortunately, someone working on the standards saw this coming – std::shared_lock and std::unique_lock are movable. Hence, we just need to adjust the idea behind the getter and setter before to instead get locks. In comes the trusty std::pair and structured bindings¹⁰:

// shared and unique lock are templated on the mutex type -- they were
// inferred before, but won't be now, so we'll just add a defaulted
// template parameter to fill them in
template <typename T, typename Mut = std::shared_mutex>
struct LockWrapper {
 // Again omitting constructors

 std::pair<std::shared_lock<Mut>, T const&> get_const() const {
 return {std::shared_lock<Mut>(mtx), obj};
 }

 std::pair<std::unique_lock<Mut>, T&> get_mut() {
 return {std::unique_lock<Mut>(mtx), obj};
 }

private:
 T obj;
 Mut mtx;
};

Now, using structured bindings we can easily get both the lock and the object:

LockedWrapper<std::string> s = "foo";

// We can append to the string
{
 auto [lock, ref_s] = s.get_mut();
 ref_s.append("bar");
}

// And we can print the string
{
 auto [lock, ref_s] = s.get_const();
 std::println("{}", ref_s);
}

This feels much better than before – we aren’t copying and we can hold the lock as long as we see fit. But, this has a bit of a semantic downside: if we want to use the wrapped object directly in an expression (not break it out into a statement beforehand, and add a scope to shorten the lifetime) we have to call the annoying .second member of the pair:

LockedWrapper<std::string> s = "foo";

s.get_mut().second.append("bar");

std::println("{}", ref_s.get_const().second);

This isn’t the end of the world, but it is a bit worse than I would’ve liked.

I dream of better (already existing) wheels

At this point what I had in my head as the dream solution almost mirrored Python’s decorators: I wanted to be able to somehow “wrap” an entire type to lock before and after like you could with a decorated function, but have that type behave almost exactly like the wrapped type – I wanted the lock to be invisible.

Now is when I reached out to a friend and much better programmer¹¹ to see what they thought; whether this was even a decent pattern, or if I was simply losing it. Turns out it was a bit of both – this was carcinization manifest.

The… crab wheel?

So, turns out, I was just inching ever closer to exactly how Rust handles this problem. Whoops.

In Rust’s std::sync they provide MutexGuard<T> – a RAII wrapper which locks an object protected by a mutex, but acts just as &T when dereferenced, and releases the lock when falling out of scope. This is exactly what I’ve wanted all along, so it’s time for the final wheel.

To mimic this, we need two new types:

One which wraps a (possibly const) reference to the wrapped type with a lock, but largely acts like the wrapped type
One which wraps a type with a shared mutex and spits out (1)

We’ll call these LockPair and Locked respectively.

`LockPair<T>`

Translating (1) to C++ land for LockPair<T>, we need to define operator*() and operator->() for some templated struct which has the templated type and a lock. When an instance of this struct is dereferenced, we want to return a (possibly const) reference to the wrapped object – this is our operator*(). The specifics of operator->() are a bit of a tangent¹², but here we can get away with just returning a (possibly const) pointer¹³ to the wrapped object.

The other half of the LockPair<T> is the lock itself. With std::unique_lock and std::shared_lock, the RAII functionality is such that when the object that currently owns the lock goes out of scope, the lock is released. Thus, we have to be sure to own the lock for this to work, and at construction we’ll want to move it in. A neat result of the RAII usage is that we don’t actually have to interact with the lock at all, we just need to hold onto it. Regardless of whether we have a unique or shared lock, any information we need to generate the appropriate methods is contained in the templated type – we can just shove the lock in a variant of the two lock types and move on.¹⁴

The final constraint on LockPair<T> to note is that we template around the wrapped type, not a reference to the wrapped type, so we’ll just add a requirement on the template that T is not a reference.

This gives us LockPair<T>:

template <typename T, typename ...Ts>
concept IsOneOf = (std::is_same_v<T, Ts> || ...);
// This is just a requirement that the input templated type is present
// in a list of types. If you're not familiar with C++'s concepts,
// just read "IsOneOf<...> auto" below as "any type in this given list
// of types"

template <typename T, typename Mut = std::shared_mutex>
 requires(!std::is_reference_v<T>)
struct LockPair {
private:
 std::variant<std::shared_lock<Mut>, std::unique_lock<Mut>> lock;
 std::reference_wrapper<T> val;

public:
 // Technically we do end up taking the lock as a forwarded reference
 // here, but the locks are only movable so we don't need to worry
 // about copying.
 template <typename S = std::remove_cv_t<T>>
 LockPair(IsOneOf<std::shared_lock<Mut>, std::unique_lock<Mut>> auto &&l,
 S &&s)
 : lock(std::move(l)), val(std::reference_wrapper<T>(std::forward<S>(s))) {
 }

 // Return a (possibly const) pointer to T
 T *operator->() { return &(val.get()); }

 // Return a (possibly const) ref to T
 T &operator*() { return val.get(); }
};

`Locked<T>`

Locked<T> has one job: control access to the contained object. Where LockPair<T> acts as a way to get to the object, Locked<T> acts as a way to get to a LockedPair<T>. Thus, we need to accomplish three main items:

Hold T
Define a way to retrieve a mutable reference to the object
Define a way to retrieve a constant reference to the object

For (1) we just need to concern ourselves with construction. If T is default-constructible, we want Locked<T> to be as well, so we’ll define that and let it be deleted whenever T is not default-constructible. In that case, we’ll have Locked<T> be initialized with some T, and once again we’ll see the forwarding reference so that we can both copy and move into Locked. Those are the options for constructing Locked<T> – once we have Locked<T> we only allow access to the wrapped object through the LockPair<T> interfaces. That is to say, there is no way to get to the contained object directly through Locked<T>.

For (2) and (3), we’ll define get_mut() -> LockPair<T> and get_const() -> LockPair<T const> which simply return {std::unique_lock(_mtx), _val} and {std::shared_lock(_mtx), _val} respectively, passing them to LockPair<T>’s relevant constructor. Once again, we instruct LockPair<T> on what kind of methods to generate (whether the accessors are const or not) based on the template parameter T, not the lock we give it. One special case we’ll want to handle is when Locked<T> itself is accessed through a const reference, say by accessing a class’s member Locked<T> in the body of a const-qualified method. In this case, we’ll want to make attempting to get mutable access a compilation error – we’ll just delete get_mut() for the const-qualified overload.

One final note on Locked<T> is that allowing T to be a pointer doesn’t really make sense, at least as far as I can imagine. For my use I don’t want to permit it to hold a pointer, so I’ll add a constraint to the template requiring this. There’s not a named constraint in <type_traits> for smart pointers, and std::is_pointer<T> does not evaluate true for them, so I have Locked<T> specialized for std::shared_ptr and std::unique_ptr with all methods deleted.

Thus, we have Locked<T>:

template <typename T>
 requires(!std::is_pointer_v<T>)
class Locked {
 mutable std::shared_mutex _mtx;
 T _val;

public:
 // Permit a default ctor only if T has one
 Locked()
 requires(std::is_default_constructible_v<T>)
 : _val() {}

 template <typename U = std::remove_cv_t<T>>
 Locked(U &&u) : _val(std::forward<U>(u)) {};

 LockPair<T> get_mut() { return {std::unique_lock(_mtx), _val}; }
 LockPair<T> get_mut() const = delete;

 LockPair<T const> get_const() const { return {std::shared_lock(_mtx), _val}; }
};

/* Render smart pointer instantiations unusable */

template <typename T> class Locked<std::shared_ptr<T>> {
 Locked() = delete;

 template <typename U = std::remove_cv_t<T>> Locked(U &&u) = delete;

 LockPair<T> get_mut() = delete;
 LockPair<T> get_mut() const = delete;
 LockPair<T const> get_const() const = delete;
};

template <typename T> class Locked<std::unique_ptr<T>> {
 Locked() = delete;

 template <typename U = std::remove_cv_t<T>> Locked(U &&u) = delete;

 LockPair<T> get_mut() = delete;
 LockPair<T> get_mut() const = delete;
 LockPair<T const> get_const() const = delete;
};

I promise it rolls

To wrap this up, I’ll just give two quick examples of this in use; one as part of an expression where we don’t want a longer living lock, and the other where we do.

For the first, I’ll pull out a piece of indy-tui’s UI where I display some information about the on track event. I want to show how much of the current event is left to go – if the current on track session is a practice or qualifying session, all that matters is the time left in the event. Conversely, if it’s a race, we should show the number of laps completed alongside the total number of laps. To accomplish this, I need to:

Determine the current session type, stored as a Locked<std::optional<Telemetry::SessionType>>
If the session type is a race:
- Retrieve the current laps completed, giving us std::optional<int>
- If the int is present, convert it to a string, otherwise do nothing
- Retrieve the total laps, giving us std::optional<int>
- Again convert to a string if present, otherwise do nothing
- Update the row as "Lap" followed by "{} / {}" formatted with the strings resulting from the above steps or "--" if not present
If the session type is a practice or qualifying session:
- Retrieve the time left in the session, stored as Locked<std::optional<std::string>>
- Update the row as "Time left" followed by the time left if present, otherwise "--:--:--"

To do this before would’ve required manually taking a shared lock of completed_laps and total_laps, possibly requiring two extra scopes if I wanted to avoid locking both at the same time¹⁵. The Table UI element that’s in use here has the ability to update a full row or column with a vector, and the std::optional<int> fields give us the ability to use the mapping methods .transform() and .and_then() to do something with the (possibly) contained value and return another std::optional<T>. Thus, if we had the ability to safely and cleanly access these fields as part of an expression, we could keep this row update concise. Thankfully, I have my trusty new (slightly oxidized) wheel:

if (sess.session_type.get_const()->value_or(
 SessionType::PRACTICE) == SessionType::RACE)
 res = table.update_row(
 1, {"Lap", std::format(
 "{} / {}",
 sess.completed_laps.get_const()
 ->transform([](auto i) { return std::to_string(i); })
 .value_or("--"),
 sess.total_laps.get_const()
 ->transform([](auto i) { return std::to_string(i); })
 .value_or("--"))});
 else
 res = table.update_row(
 1, {"Time left", sess.time_to_go.get_const()->value_or("--:--:--")});

For that second example, I’ll pick out a smaller component¹⁶. In the logic to display the live leaderboard, I have a table of possibly changing size dependent on two things – the number of drivers, and the number of columns desired¹⁷. Thus, when I want to update one of these values, I need to do some reconfiguration afterwards.

For some reason, I decided to have setters for the number of drivers and the columns individually, pulling the reconfiguration out to another method which both call. Multiple operations are done assuming all properties are unchanged throughout the reconfiguration, so all will need to be locked in some way. These don’t change much, so it wasn’t too much of an issue to grab a unique lock on each element (the table, the vector of columns, and the number of drivers). As discussed earlier though, I needed to be sure we’re not releasing the lock on the newly changed component – I decided to move the retrieved LockPair into the reconfiguration call. For the other components, I didn’t want the reconfiguration method to have to know what locks it needs to take based on the caller, so I decided to simply have the reconfiguration method move in a LockPair<T> for each of the three components. This way, the caller is the arbiter of which locks are taken when, moving them all into the reconfiguration at the end regardless.

For updating the columns and the subsequent reconfiguration this gives us:

void LeaderboardView::set_columns(std::vector<ColumnInfo> &&cs) {
 auto locked_columns = _cols.get_mut();
 *locked_columns = std::move(cs);

 reconstruct_table(table.get_mut(), std::move(locked_columns),
 _num_drivers.get_mut());

 // We no longer own the lock on the columns, so even though
 // locked_columns goes out of scope, the lock is not released
}

void LeaderboardView::reconstruct_table(
 LockPair<UI::Table> &&locked_table,
 LockPair<std::vector<ColumnInfo>> &&locked_columns,
 LockPair<size_t> &&locked_num) {

 // Reset dimensions, adding 1 row to account for column titles
 locked_table->set_dim(Table::Dim(*locked_num, locked_columns->size()));

 // Set alignment based on the columns
 for (auto [idx, c] : std::views::enumerate(*locked_columns))
 locked_table->column_props(idx)->align = c.align;

 // All three locks are going to go out of scope, and we own them
 // here so each lock will be released
}

Conclusion

The material takeaway is the above – Locked<T> and LockPair<T>. But, further than that, this whole endeavor was a bit of an excursion. Personally, I love to take these sorts of things on. It’s mostly simple, which means nailing the ergonomics (and actually getting the code right) is that much more important; if the thing I’m trying to smooth out winds up easier to use than the thing I make… well at least I have something to learn from.

Likewise, they also give me a chance to really grow my knowledge. If I’m trying to just get a task done, I’m going to figure out how to do it and probably move on. Conversely, if I’m trying to make something that I plan to use generally, I have to take more time paying attention to edge cases and what’s really going on. Something that directly preceeded this actually an attempt to marry this concept with std::optional, making some sort of LockedOptional. Even though that was very scrapped, trying to hand-roll std::optional was very, very interesting and was a great learning experience¹⁸.

So, did I reinvent the wheel a tad bit? Maybe. But, I left with a deeper understanding of C++ than I went in with, and now I have a tool which has already proved very helpful.

Epilogue – These wheels are made for rolling

One final not-entirely-relevant note which is relevant enough to mention.

Very quickly after finishing this implementation, I noticed LockPair<T> is actually useful beyond being just a conduit for Locked<T>. Elsewhere in the project, I have shared components which are a little more complex than Locked<T> feels good for. In one such case, I have a frequently rearranged vector of driver objects which themselves have a fair amount going on. Between this and existence of implementation that I don’t see the benefit in updating, this is a shared component which still must be manually locked.

However, as part of a refactor, I needed to give a part of the UI a way to get a snapshot of this vector, as it’s the current ordering of the drivers on track. These drivers objects are not exactly the smallest, and this is one of the most frequently hit sections of the codebase, so I don’t want to be copying the vector each time. Ultimately, what I wanted was a way to hand off a const reference to this vector with an automatically handled lock.

Hey wait that sounds familiar.

Using LockPair, I can do exactly that, even though the vector and its mutex are not related to a Locked<T> object whatsoever. Thus, the call for the UI to retrieve the drivers in their on-track ordering is roughly:

ThreadSafe::LockPair<std::vector<DriverTelemetry> const>
TelemetryBoard::reorder_and_get() {
 // Order drivers by rank, then calculate gap to leader
 {
 std::unique_lock lock(_driver_vec_mtx);
 /* ... Omitted as it's not relevant ... */
 }

 // Acquire and return a shared_lock on the drivers vec
 return {std::shared_lock(_driver_vec_mtx), _drivers};
}

See the github repo for the project, or #indy-tui for more here. ↩︎
Huge shoutout to this wonderful talk from David Olsen Back To Basics: C++ Concurrency. ↩︎
A tip. If you are early on in a project and are already in enough of a deadlock mess to think that logging lock acquisition is going to be helpful. Maybe like, do something else? Oops. ↩︎
We need not discuss the caveat. It is certainly not a queue. Certainly I did not try a thread-safe queue based on what I remember from my computer organization class. Certainly. ↩︎
Read: mutilation. ↩︎
I did not end up trying a macro. The world should consider itself lucky there is a part of my brain capable of keeping me from reaching for macros and goto. ↩︎
Don’t call me Shirley. ↩︎
Fear not for later that day I watched Casey Muratori’s The Big OOPs and that concluded. ↩︎
To verify, I wrote up a simple compiler explorer example – as far as I can tell, GCC and Clang seem to do different things with this set() then get() sequence. In GCC I cannot get the two to disagree (i.e. another set() occurs in between pair set() and get() calls), but in Clang it seems relatively frequent. I might investigate this further later, but here’s the link: https://godbolt.org/z/hqWrfPve3. ↩︎
I can’t find a link to the standards discussion but someone is proposing nested structured bindings which would be so good please I am begging. ↩︎
Huge shoutout to Rose please check out their page https://ikl.sh/. ↩︎
And tangent I will. If the operator->() overload returns a pointer, it behaves exactly as expected, giving us the equivalent of (*obj).foo(). But, if it returns a non-pointer, operator->() is then applied on that value, repeating this until a pointer is returned, which is then dereferenced as usual. This “drilling down” behavior was (to me) surprising but I guess makes sense – if we have T** ptr and want to access the object, it would be quite annoying to have to do ptr->->foo(). ↩︎
An earlier attempt saw me returning a pointer whenever the wrapped object did not have the dereference operator defined, otherwise I returned a reference to the result of dereferencing it. Because of that drill-down behavior this actually punched straight through things like std::optional and would’ve been a nightmare… ↩︎
I’ve just now realized that, if desired, we could further template this to allow us to define the possible lock types, meaning we could extend this past shared and unique lock if we so desired. ↩︎
Which I’d want to do to avoid unnecessary contention as these would be updated often. ↩︎
What’s here is already a bit vestigial – I need to update some pieces which would wind up mooting this part. But it’s a good example. ↩︎
The columns are functors which produce a string given a driver’s current information, so adding a new column of information is simply adding a new functor. ↩︎
I have read Microsoft’s deducing this article so many times, but only after recognizing that my four overloads of every method was exactly what the author was talking about did it really click. ↩︎

Delayable Telemetry Queue | Design Notes - indy-tui

foo Thu, 06 Aug 2026 08:23:40 -0400

As the core impetus for my most recent (and ongoing) project, I think a break down of my design and implementation of the delay queue used in #indy-tui is perfectly befitting of a first post.

I’ll start with a little background on the project, follwed by a little more on the problem I was trying to solve and why I’m/ taking a crack at it. Then, I’ll touch on the two main approaches I considered, what primary goals I was trying to meet, and ultimately which of the approaches stuck. Finally, I’ll go a little more technically in-depth on the implementation I have today.

If you are a motorsports fan, the non-technical part(s) of this post should be right up your alley. If you aren’t, fear not – the aforementioned non-technical part(s) should be quite limited. With that said, why not just… become one? I mean it’s pretty cool¹.

This is a solution, but certainly not the solution – my hope is to illustrate why I picked this one.

Background

The problem

Compared to “stick and ball” sports like hockey, football, or soccer, broadcasting motorsports presents a unique challenge: the broadcast can only ever show a very small fraction of what’s happening at any given time. Aside from the primary challenge that it’s just not possible to show all 25+ cars spanning miles on screen at once, each of those cars has a team plotting strategy (sometimes) on the radio throughout the duration of the event. Needless to say, if your favorite driver is not in the top X or is clearly charging through the field, it’s not a given they’ll get much coverage (if any).

To their credit, various broadcasts have made progress trying to tackle this problem. The most important of these spectator assist devices is inarguably the on screen leaderboard – a live graphic which displays the state of the race (flag status, laps completed/laps to go) and, at minimum, the current position of each driver in the field. When used more effectively, the broadcaster will include more live information on the leaderboard, data points like each driver’s gap to the leader, the number of laps since they last pitted, or the tire compound they’re currently equipped with.

For the more observant viewer, the leaderboard goes a long way in filling the gaps inherent to motorsports coverage. But, for the more impatiently observant viewer², this presents a new pain point in the broadcast’s production: rarely is the leaderboard showing the information you would like to know at any given time. Thankfully, today most premier motorsports series provide a much more thorough live leaderboard separate from the broadcast, often by app, which displays most of the information at once. With this alternative, said impatiently observant viewers² should be satisfied – they have all of the information when they want it, even during commercials. So… they are satisfied right? Right?

Have you ever had an upstairs neighbor who somehow has a slightly faster stream of the football game that you both are watching? Watched a pass thrown and had all suspense robbed from you as said neighbor is celebrating the touchdown when, for you, the ball is still in the air? Unfortunately, unless you are lucky enough to somehow³ have a broadcast with next to zero delay, this upstairs neighbor and these online leaderboard options are one in the same – the online leaderboards have a very neglible delay, while broadcasts do not. Using what the series gives you, your two options are to either just take the on screen leaderboard and accept the lack of spoilers over getting the information you want, or to use their online leaderboard and get the information desired but spoilers along with it.

All that’s really needed here is a way to delay the leaderboard information (from here on “telemetry”) by a configurable amount. While I don’t know of any major series which offers this directly, there is third party tooling which does⁴. Although the options I’ve tried out do work, I’ve had a few main gripes:

The delay is configured in milliseconds. When I’m trying to measure a delay while watching a race, I personally don’t want to bother doing the math of adding the right number of zeros to the rough number I’ve observed. At most, I want control down to half-seconds⁵.
When accruing a delay, the interface stalls and doesn’t communicate whether it is broken, there is no telemetry coming in, or it is simply building up a delay. I would like a small portion of the interface to show the status of the delay, for example how much of a delay is currently built up.
The delay overshoots or undershoots over long spans. I have observed times where after adjusting the delay back and forth, it is not uncommon to wind up with the observed delay markedly longer than the configured delay.
The tools are not open source. Ultimately, this is the most consequential gripe – if I am the only person in the world that has the above issues, I will happily fork the project and make it most comfortable for myself. Without that ability, I am left to either suck it up or try my hand at a solution.

So, try my hand I did.

Goals

The delay is consistent.
- There should be strong guards against undershoot and overshoot.
Exact delay is more important than continuity of telemetry.
- There will be stutters in the telemetry we receive. I care that the duration between front and back of the queue matches the desired delay much more than I can that the telemetry shown is continuous (w/o skipping)
The delay is configured in second increments.
- If implemented well, this is easily updated later to be half-second (or smaller) increments.
The queue is thread-safe.
- When used in the project, we’ll want to have what is basically a producer-consumer setup.
Delay status is retrievable.
- We’ll want to show on the interface whether the delay is satisfied, and if not how much of a delay is accrued.
The delay can be zero.
- If the user wants a “true” live feed, we don’t want to have a second queue to accommodate it.

Two approaches

When first tackling this problem, two main solutions jumped out at me – a dynamic queue, and a fixed-size queue.

Dynamic queue

The first, and what I suspect most other tools attempting this use, is a straightforward thread-safe queue where the “producer” thread enqueues receieved telemetry paired with a timestamp, and the “consumer” thread dequeues telemetry frames only if the duration between now and that frame’s timestamp satisfy the delay.

This solution satisfies goals three, four, five, and partially satisfies goal one as undershooting would be impossible. This solution also benefits from a fairly straightforward implementation, likely requiring minimal work on top of the C++ standard library.

A byproduct of this implementation is that as long as we receive a given telemetry frame, the user will “see”⁶ every frame. Note that I described this as a byproduct rather than a benefit – this actually violates goal two. It is unavoidable that the telemetry we receive will be at an inconsistent rate, so by ensuring this continuity we run the risk of a delay overshoot (thus also violating the other half of goal one). With this solution, we end up implementing a “best case” delay; if we receive telemetry faster than we can process and display it, we will quickly build up a surplus of unprocessed telemetry frames, resulting in a delay overshoot.

If the end goal were a data collection tool for later analysis of race telemetry (which wouldn’t require delay functionality) this continuity would likely be paramount, and this solution would certainly win out. But, that’s not our application, so we need another plan.

Fixed-size queue

This fixed-sized queue leverages two truths in this specific application: TUI users don’t expect the utmost temporal resolution for data presentation, and only so much data is actually understandable to humans at once. That is to say, in this application, there is quickly a limit to how fast the interface’s refresh rate actually needs to be. Thus, we can fix a refresh rate, for now at 10 Hz.

Given a fixed refresh rate and a configured delay, we can fix the size of the queue in total delay frames required as the refresh rate multiplied by the desired delay. Then, we allow dequeue only when all frames are filled, and we allow enqueue only when not all frames are filled. Alongside that, the number of frames between the front and back gives the total delay currently accrued.

As long as we know that we can always process and display an individual frame faster than the refresh⁷, we know that the delay is consistent as per goal one. Similarly, by only allowing enqueue when the delay is not satisfied, we know that at any given time the frame at the back of the queue is a snapshot in time exactly as long ago as the delay is configured, regardless of whether there were received frames between the last “snapshot” and the most recent one, satisfying goal two. The “fixed” queue size is using the configured delay, so regardless of the current choice we can recalculate it as the configuration changes, satisfying goal three. As you will see, the implementation is thread safe, and as noted above we can easily calculate the current status of the delay, knocking out goals four and five.

This leaves us with goal six, and there’s an option here to feed two birds with one scone⁸. See, one problem with this proposed solution is a sort of “hitching” problem – there’s this back-and-forth dance where the queue is full, the telemetry frame at the front is dequeued and shown, and before a new frame is received the interface tries and now fails to dequeue the next frame. To get around this, we can simply add some “slop” to the end of the queue, some relatively small number of frames with which we permit the queue to just barely over/undershoot. This way, the interface has a handful of delay-satisfactory frames to chew on while waiting for new telemetry to come in, and the telemetry receiver thread has some padding to flex in case a burst of frames come in.

Now we feed the second bird: when we want to go “live”, we drop the queue down to just the “slop” frames. With a refresh rate of 10 Hz (that’s 10 telemetry frames every second) it would take ten slop frames to reach a delay of one second. So, as long as we keep the slop frames to at most around five, we can keep the “live” delay down to less than half a second. This does reintroduce the hitching problem, but with a truly live feed the hitching is unavoidable as again the telemetry is coming in at a varying rate. As far as I have observed in testing, this solution is perfectly satisfactory for a live display.

That’s all six goals – on to implementation.

Implementation

Note: This will reflect what is live in the repo at the time of writing, but I do not guarantee that it will remain that way.

The `DelayInfo` dataclass

Whenever either delay_s (delay in seconds) or refresh_hz (refresh rate in hz), two calculated parameters change:

total_frames : The calculated number of frames in the queue
frame_period : The amount of time represented by one frame

Given how tightly bound these four parameters are, and how frequently they’re used together throughout the implementation, it’s important for thread safety that we prohibit the use of one whilst any other is changing. This is all DelayInfo exists to do.

Omitting some not-so-relevant details, we lay out the class as:

class DelayInfo {
 static constexpr size_t DELAY_SLOP = 4;

private:
 size_t _delay_s = 0;
 size_t _refresh_hz = 10;

 std::shared_mutex _mtx;

 void recalculate_params();

public:
 /** @brief Instantiate the delayinfo with the configured parameters. */
 DelayInfo(size_t delay_s, size_t refresh_hz)
 : _delay_s(delay_s), _refresh_hz(refresh_hz) {
 recalculate_params();
 }

 void set_delay_s(size_t const);
 void set_refresh_hz(size_t const);

 [[nodiscard]] size_t get_delay_s();
 [[nodiscard]] size_t get_refresh_hz();
 [[nodiscard]] size_t get_delay_frames();
 [[nodiscard]] Time::Duration::DblMilliSec get_frame_period();
};

The most important detail here is the shared_mutex. Since recalculate_params will need to change multiple paramaters, we need to guard against any getters/setters accessing any parameters during recalculation. But, since the getters don’t need to change any state and thus don’t need to block other getters, we simply take a unique_lock of _mtx in the setters and recalculate_params, and take a shared_lock in the getters. Given how relatively infrequently the delay configuration should be changing, _mtx should rarely be contested.

We’ll skip going over the implementation of the getters and setters, as recalculate_params should give a good idea:

void recalculate() {
 // Called by any setters
 std::unique_lock lock(_mtx);

 _total_frames = (_delay_s * _refresh_hz) + DELAY_SLOP;
 _frame_period = Time::Duration::DblMilliSec(
 1000.0 / static_cast<double>(_refresh_hz));
}

Here, we:

Take the unique_lock as mentioned before
Calculate the number of frames needed as the delay multiplied by the refresh rate, adding the slop frames
Calculate the frame period (total amount of time represented by one frame, in milliseconds) as the reciprocal of the refresh rate

`TelemetryQueue`

Rather than just walk through the class definition, we’ll approach it from a more bottom-up direction. To start with, we’ll go over some background on the queue and then the most important two functions of this (or any) queue: enqueueing and dequeueing. Following that, we’ll close out by covering the queue reconfiguration process.

Queue background

Since the number of the frames in the queue at any time is essentially fixed-size, I opted to reach for a single-ended circular queue. With this approach, we simply track monotonically increasing enqueue and dequeue indices, adjusting them only when we take in or give out a telemetry frame. Then, to access the queue, the actual position is calculated as the index reduced modulo the queue size.

As highlighted earlier, a primary goal was to be able to split the “producer” and “consumer” logic between different threads; one to receive telemetry, the other to process and display it. With this design, we just need two small additions to get there:

A mutex on each individual frame, so that we don’t try to access the same frame in both threads (possible if the queue is either empty or full)
A mutex on the entire queue, so that we can lock all frames during resize (more on this shortly)

One final piece to note is my error handling strategy. Whenever possible, I prefer to reach for “errors as values” as opposed to exceptions, and I do this for two primary reasons:

Forcing the caller to acknowledge the existence of and do something with errors makes code more readable at the callsite (in my opinion).
Requiring erroneous paths to return meaningful information forces me to more carefully consider what should be an error and often leads me to write more thorough code with stronger guarantees.

This approach is what I chose for most of the error handling in this project. Here, I have an error type TelemetryQueue::Err which has various states representing errors we might expect to see. Here, the two most relevant states are Err::TOO_RECENT and Err::DELAY_FULL, noting respectively that there is currently not enough accrued delay to dequeue a frame or that there is currently exactly enough accrued delay and to enqueue a new frame would cause the delay to overshoot.

With that, we’ll now walk through the implementation.

Dequeue

To start with, we see the aforementioned errors-as-values approach. Here, we leverage C++23’s wonderful std::expected, where we note that we’ll return either a unique_ptr representing a telemetry frame, or an Err representing the error encountered. Also, since we are handling errors using std::expected, we will not throw any exceptions and anything we call will not either. Thus, we can mark the method as noexcept.

std::expected<std::unique_ptr<TelemetryFrame>, TelemetryQueue::Err>
TelemetryQueue::dequeue() noexcept {

First, we double check that the dequeue index is not ahead of the enqueue index, as this is considered “impossible” – if this has happened, we are in an unrecoverrable state. As such, we check this with an assertion⁹:

assert(_deq_idx <= _enq_idx);

Next we check that the delay is satisfied. The call to delay_info’s get_min_delay_frames() just retrieves the minimum number of frames between the enqueue and dequeue indices such that the delay is satisfied, which is just the total number of frames before the addition of the slop frames. If the delay is satisified, we continue on, and if not, we log a debug message and return the corresponding error state

if ((_enq_idx - _deq_idx) >= _delay_info.get_min_delay_frames()) {
 // ... what follows goes here ...
} else {
 Tools::Log::Debug("Dequeue skipped, delay is not ready", "TELEM-QUEUE");
 return std::unexpected(Err{Err::TOO_RECENT});
}

Continuing assuming the delay is satisfied, we get a shared_lock on the full queue and then again check against an “impossible” state to ensure the queue is not empty. The mutex on the full queue will only be contested if we try to resize the queue, a relatively rare occurence. Every step following assumes that the queue has not been resized, so we acquire the shared_lock (using a RAII wrapper, so .lock() is called at construction and .unlock() at destruction) at the start and do not release until the true body exits. Since this is a shared_lock, both enqueue and dequeue can acquire it simultaneously.

std::shared_lock full_lock(_frame_mtx);
assert(!_frames.empty());

Next we index the full queue at the dequeue index reduced modulo the queue size, and acquire a lock on its mutex (this is again a RAII wrapper, just not shared lock this time).

auto &cur_frame = _frames.at(_deq_idx % _frames.size());

std::scoped_lock single_lock(cur_frame.mtx);

Now we check if the frame retrieved is actually present. Given that we return frames as a std::unique_ptr<TelemetryFrame>, when we give them to the caller we are moving¹⁰ them out of the queue. Then, since std::unique_ptr’s move constructor sets the moved-from ptr to nullptr, a frame which is not yet populated will be nullptr. As such, if the frame we retrieve is nullptr, it means that we have somehow advanced to a frame which is not ready, so the delay is not yet satisied. This should generally not happen, but it is good practice to check pointers regardless.

The use of std::unique_ptr for handing around the frames is a vestige of prior unrelated troubleshooting regarding the TelemetryFrame’s move constructor. At this point it’s not necessary, and is probably a target for future refactor/cleanup. With that said, the indirection cost of a pointer when accessing the frame is negligible here, so to change it would be only for cleanliness (still a very worthy cause).

if (cur_frame.frame == nullptr)
 return std::unexpected(Err{Err::TOO_RECENT});

Finally, we know that we have a valid frame. As such, we’ll step the dequeue index forward one frame and then return the frame we retrieved before.

_deq_idx++;

return std::move(cur_frame.frame);

Now for the full dequeue logic:

std::expected<std::unique_ptr<TelemetryFrame>, TelemetryQueue::Err>
TelemetryQueue::dequeue() noexcept {
 assert(_deq_idx <= _enq_idx);
 if ((_enq_idx - _deq_idx) >= _delay_info.get_min_delay_frames()) {
 std::shared_lock full_lock(_frame_mtx);

 assert(!_frames.empty());

 auto &cur_frame = _frames.at(_deq_idx % _frames.size());

 std::scoped_lock single_lock(cur_frame.mtx);

 if (cur_frame.frame == nullptr)
 return std::unexpected(Err{Err::TOO_RECENT});

 _deq_idx++;

 return std::move(cur_frame.frame);
 } else {
 Tools::Log::Debug("Dequeue skipped, delay is not ready", "TELEM-QUEUE");
 return std::unexpected(Err{Err::TOO_RECENT});
 }
}

Enqueue

Since the enqueue logic mirrors that of dequeue, we’ll start with the full method and dissect where they differ.

std::expected<void, TelemetryQueue::Err>
TelemetryQueue::enqueue(std::string_view const payload) noexcept {
 assert(!payload.empty());
 assert(_deq_idx <= _enq_idx);

 if ((_enq_idx - _deq_idx) < _delay_info.get_delay_frames()) {
 std::shared_lock full_lock(_frame_mtx);

 assert(!_frames.empty());
 auto &cur_frame = _frames.at(_enq_idx % _frames.size());
 std::scoped_lock single_lock(cur_frame.mtx);

 cur_frame.frame =
 std::make_unique<TelemetryFrame>(Tools::b64_decode(payload));

 _enq_idx++;

 return cur_frame.frame->is_valid()
 ? std::expected<void, TelemetryQueue::Err>{}
 : std::unexpected(Err{Err::FRAME_INVALID});
 } else {
 Tools::Log::Debug("Enqueue skipped, delay is full", "TELEM-QUEUE");
 return std::unexpected(Err{Err::DELAY_FULL});
 }
}

Firstly, here we take in a string_view of the received base64-encoded telemetry payload. We only need to read the string we take in, and that string’s lifetime will extend beyond this called function, so we use a string_view and save on an unnecessary copy. We’ll also double check that payload is non-empty with an assertion, as this should not be possible.

The second difference is in the delay satisfaction check – this is the negation of the one in dequeue, thus we only permit enqueueing a new frame if there are fewer frames than the delay requires between the front and back.

Next, rather than retrieve a frame we must create one. We do this by first getting a reference to the current slot in the queue, and then construct a frame by passing in the base64-decoded payload.

The final difference is in the return: if we succeed, we don’t need to return anything, and if we don’t we need to tell the caller why. Hence, once we’ve built the frame and set the current slot in the queue accordingly, we need to check if the frame is in a valid state. If it is, we return nothing, and if not, we return the previously mentioned Err with the corresponding state.

Queue reconfiguration

Here is where we’ll handle rebuilding the delay upon delay reconfiguration. If the delay is decreased, we need to shrink the queue size, and likewise grow it if the delay is increased.

It would be a very frustrating user experience if the full delay had to be re-accrued upon every delay change, so we need to preserve whatever amount of the delay is relevant when rebuilding the delay – we will come back to this shortly.

As mentioned before, delay reconfiguration is the only time the full queue’s mutex needs to block enqueue/dequeue (and enqueue/dequeue needs to block reconfiguration), so we’ll start by taking a unique_lock on the full queue mutex and a scoped_lock on the DelayInfo’s mutex.

void TelemetryQueue::rebuild_delay() noexcept {
 std::unique_lock frame_lock(_frame_mtx);

 std::scoped_lock lock(_delay_info_mtx);

Next we calculate the new queue size, and then how many frames forward we need to skip. If the delay is increasing, the frame at the front of the queue (next to be dequeued) is now too recent, and thus no skip is necessary. If the delay is decreasing, the front of the queue is now too old, requiring us to skip some number of frames ahead.

size_t prev_delay_frames = _frames.size();
size_t new_delay_frames =
 _delay_info.get_delay_frames();

size_t skip_frames = (new_delay_frames < prev_delay_frames)
 ? ((prev_delay_frames - new_delay_frames) - 1)
 : 0;

Since C++’s std::mutex is neither copyable nor movable, and the queue contains a wrapped version of TelemetryFrame adding a mutex, the std::vector which underlies the queue is also neither copyable nor movable. Hence, we cannot simply extend the existing vector – we instead create a new one and move preserved frames over.

So, first we create the new vector with the new length:

std::vector<LockedFrame> new_frames(new_delay_frames);

Then, if this is the initial configuration of the queue, there are no frames to preserve, and we can just set our vector and return early:

if (_enq_idx == 0) {
 _frames = std::move(new_frames);
 return;
}

Next we move the dequeue index forward by the number of skip frames calculated before. When the delay is rapidly increased and decreased, it is sometimes possible to end up with the skip frames pushing the dequeue index ahead of the enqueue index. Since the dequeue index should always be less than or equal to the enqueue index, we’ll set it to the minimum of the two:

_deq_idx += skip_frames;
_deq_idx.store(std::min(_deq_idx, _enq_idx));

Now we’ll copy over the frames we need to preserve, if any. Starting at the dequeue index, we’ll simply move each frame over until we reach the enqueue index:

if (prev_delay_frames > 0) {
 for (size_t copy_idx = _deq_idx; copy_idx < _enq_idx; ++copy_idx) {
 new_frames.at(copy_idx % new_delay_frames).frame =
 std::move(_frames.at(copy_idx % prev_delay_frames).frame);
 }
}

Finally, we move the new vector into _frames and are done:

_frames = std::move(new_frames);

In total, for queue reconfiguration we have:

void TelemetryQueue::rebuild_delay() noexcept {
 std::unique_lock frame_lock(_frame_mtx);

 std::scoped_lock lock(_delay_info_mtx);

 size_t prev_delay_frames = _frames.size();
 size_t new_delay_frames =
 _delay_info.get_delay_frames().value_or(DelayInfo::DELAY_SLOP);

 size_t skip_frames = (new_delay_frames < prev_delay_frames)
 ? ((prev_delay_frames - new_delay_frames) - 1)
 : 0;

 std::vector<LockedFrame> new_frames(new_delay_frames);

 if (_enq_idx == 0) {
 _frames = std::move(new_frames);
 return;
 }

 _deq_idx += skip_frames;
 _deq_idx.store(std::min(_deq_idx, _enq_idx));

 if (prev_delay_frames > 0) {
 for (size_t copy_idx = _deq_idx; copy_idx < _enq_idx; ++copy_idx) {
 new_frames.at(copy_idx % new_delay_frames).frame =
 std::move(_frames.at(copy_idx % prev_delay_frames).frame);
 }
 }

 _frames = std::move(new_frames);
}

Conclusion

There is much, much more to talk about regarding this project; implementing a delay was merely the main motivating factor in me starting it. The design I originally pictured and what is live on GitHub certainly differ, but not by that much. Even so, there were countless different iterations at what you see here, all of which will forever remain as commits quickly undone.

I started with something resembling what you see here, albeit significantly less “developed” and explainable. With many tweaks and corrections, I was eventually able to get that initial design to (mostly) work. But, as I’m sure many can relate to, I convinced myself that wasn’t the “right” way to tackle this problem. Somewhere along the way I had convinced myself that I was “cheating”, that this was a hacky solution – I had myself convinced that if I wanted to do it “right” I needed to implement the aforementioned dynamic queue. I sunk a not insignificant amount of time into the dynamic solution, got it to the point where it worked with my crude prototype of a display, and excitedly got to testing it.

It did not work.

The delay was inconsistent, it wasn’t always clear whether the delay was even building correctly. Every time I’d try to use it, I would find a different way to trick the delay into getting out of sync. To be sure, a fair amount of these problems originated between the keyboard and chair, but some of the failure modes I encountered were remarkably reminiscent of the ways I’ve seen other similar tools fail.

Whether any subset of the issues I noted using the dynamic solution were inherent to that solution or not, I had to make a call: stick with what I had convinced myself to be the “right” solution which I could not seem to make work, or go back to what I had come up with on my first pass. Put more bluntly, I needed to decide whether to get in line with what I had convinced myself “everyone else” was doing, or to trust my design intuition.

As now explained, I went with the latter. After much consideration, I’d decided that I could more effectively defend the solution seen here than the dynamic solution I had laid out. I went back and once again implemented yet another telemetry queue, now taking a little more time to make sure every decision was grounded in a set of end goals. What resulted was definitely rough around the edges and took some effort to smooth it out – that’s unavoidable. But, with that effort, I eventually got to a point where I finally couldn’t trick the delay into failing. Surely, there are latent issues which will show themselves in due time, but what I have now is something that meets the goals I laid out, built following a design process I executed.

Once again: this is a solution, certainly not the solution. Hopefully, it’s now clear why I’ve picked this one.

yeah this was lowkey track limits but it’s still awesome ↩︎
it’s me I am said viewer ↩︎ ↩︎
this is a really cool explanation of this delay problem and why it’s not going anywhere https://www.youtube.com/watch?v=CgcXli8NxHw ↩︎
multiviewer is an excellent tool and works across many series ↩︎
Also, your observed broadcast delay fluctuates over time – if you want calibrate your delay to less than 0.5s/500ms you’re likely going to have to spend a lot of time monitoring and readjusting it. I personally would like to just set my telemetry delay to be just longer than my broadcast delay and accept that what I see on my leaderboard is just barely behind what I see on TV. ↩︎
telemetry is sent rather quickly, with each frame representing at most slightly less than a second ↩︎
and, we can – we pick the refresh rate ↩︎
this has, for better or worse, worked its way into my lexicon: https://x.com/peta/status/1070066047414345729 ↩︎
I cannot wait to rewrite this and make this a pre and post condition once C++26 is mature ↩︎
here is a decent explainer on those new to C++’s move semantics: https://stackoverflow.com/questions/3106110/what-is-move-semantics ↩︎

schelcc | blog on schelcc

C++ Thread Safety, Reinventing The Wheel, and Making My Life Easier

Background#

I like dataclasses#

The triangular wheel#

The hexagonal wheel#

I dream of better (already existing) wheels#

The… crab wheel?#

LockPair<T>#

Locked<T>#

I promise it rolls#

Conclusion#

Epilogue – These wheels are made for rolling#

Delayable Telemetry Queue | Design Notes - indy-tui

Background#

The problem#

Goals#

Two approaches#

Dynamic queue#

Fixed-size queue#

Implementation#

The DelayInfo dataclass#

TelemetryQueue#

Queue background#

Dequeue#

Enqueue#

Queue reconfiguration#

Conclusion#

Background

I like dataclasses

The triangular wheel

The hexagonal wheel

I dream of better (already existing) wheels

The… crab wheel?

`LockPair<T>`

`Locked<T>`

I promise it rolls

Conclusion

Epilogue – These wheels are made for rolling

Background

The problem

Goals

Two approaches

Dynamic queue

Fixed-size queue

Implementation

The `DelayInfo` dataclass

`TelemetryQueue`

Queue background

Dequeue

Enqueue

Queue reconfiguration

Conclusion