[lttng-dev] [rp] [RFC] Readiness for URCU release with RCU lock-free hash table

[Paul E. McKenney]

On Tue, May 01, 2012 at 11:12:15AM -0400, Mathieu Desnoyers wrote:
> * Paul E. McKenney (paulmck at linux.vnet.ibm.com) wrote:
> > On Tue, May 01, 2012 at 10:16:09AM -0400, Mathieu Desnoyers wrote:
> > > Hi!
> > > 
> > > After 1 year of development, with the last 6-7 months spent polishing
> > > the API and testing the implementation, I think it is getting about time
> > > to release the RCU lock-free hash table in a new Userspace RCU version
> > > (0.7).
> > > 
> > > I recently described the guarantees provided by the hash table in more
> > > detail, and created tests for the uniqueness guarantee for traversals
> > > performed concurrently with add_unique and add_replace operations. I
> > > also added test modes that create long hash chains, to test corner-cases
> > > of the hash table.
> > > 
> > > One small thing I wonder is whether we should document that the hash
> > > table update operations imply full memory barriers ?
> > 
> > The Linux kernel's rule seems good here -- if a hash-table operation is
> > atomic and returns a value, it should imply a full barrier.  So:
> > 
> > cds_lfht_new(): No point in claiming barriers -- publishing the
> > 	pointer to the hash table is where the barriers are important.
> > 
> > cds_lfht_destroy(): Ditto.
> > 
> > cds_lfht_lookup(): Not an update (let alone an atomic update), no barriers.
> > 
> > cds_lfht_next_duplicate(): Ditto.
> > 
> > cds_lfht_first(): Ditto.
> > 
> > cds_lfht_next(): Ditto.
> > 
> > cds_lfht_add(): Atomic update, but no return value, so no barrier
> > 	implied.
> 
> Yep, makes sense. We use cmpxchg internally to perform the update, but
> it could make sense to eventually use a cmpxchg that has no memory
> barriers to perform this update. So I agree on not providing a memory
> barrier guarantee on the "add" operation, since it does not return any
> value.
> 
> > cds_lfht_add_unique(): Atomic update that returns a value, so should
> > 	imply a full memory barrier.
> 
> add_unique is a bit special:
> 
> - if it returns the node received as parameter, it means the add
>   succeeded, which imply an update, and thus a memory barrier.
> 
> - if it returns a different node than the one received as parameter, it
>   failed, and thus means that it only performed a lookup, so there is no
>   guarantee that a memory barrier has been executed.

Good point!  But I would suggest handling this like some architectures
handle a failing cmpxchg() -- there is a memory barrier beforehand,
but not afterwards.  This works well because although the caller can
easily supply a memory barrier after a failing cmpxchg(), there is no
way to supply one beforehand except by supplying a redundant barrier in
the case of a successful cmpxchg().

So I suggest that a failing add_unique() guarantee full memory barrier
beforehand, but not afterwards.

							Thanx, Paul

> > cds_lfht_add_replace(): Ditto.
> > 
> > cds_lfht_replace(): Ditto.
> > 
> > cds_lfht_del(): Ditto.
> 
> Yep, makes sense.
> 
> I'll add this documentation in the API.
> 
> Thanks!
> 
> Mathieu
> 
> > 
> > cds_lfht_is_node_deleted(): Not an update (let alone an atomic update),
> > 	no barriers.
> > 
> > cds_lfht_resize(): Atomic update, but no return value, so no barrier
> > 	implied.
> > 
> > cds_lfht_for_each(): Not an update (let alone an atomic update),
> > 	no barriers.
> > 
> > cds_lfht_for_each_duplicate(): Ditto.
> > 
> > cds_lfht_for_each_entry(): Ditto.
> > 
> > cds_lfht_for_each_entry_duplicate(): Ditto.
> > 
> > 							Thanx, Paul
> > 
> > > I'm personally getting confident that the hash table API is clean
> > > enough, and the implementation well tested, but I'd like to have your
> > > thoughts on the readiness of the hash table for production use.
> > > 
> > > Thanks,
> > > 
> > > Mathieu
> > > 
> > > -- 
> > > Mathieu Desnoyers
> > > Operating System Efficiency R&D Consultant
> > > EfficiOS Inc.
> > > http://www.efficios.com
> > > 
> > > _______________________________________________
> > > rp mailing list
> > > rp at svcs.cs.pdx.edu
> > > http://svcs.cs.pdx.edu/mailman/listinfo/rp
> > > 
> > 
> > 
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev at lists.lttng.org
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> -- 
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
>