Plugins: Super Forum Dictionary Key

[Peter]

I know that the v6 JavaScript API hasn't been discussed yet, but I thought it was time to start throwing a few feature suggestions out there now in the hope that if accepted, they get implemented for the release.

Unless v6 is going to allow us to pull fresh data when we like, then we need solutions to prevent dirty data getting set in the key. This problem is why some plugins haven't been created yet.

---

I would like to have a new key type that will be a dictionary (key/value pairs).

This type of key would allow for a specific entry in the dictionary to be updated only.

An example of how this key could be used is for a type of Auction plugin.

• - Admin creates an Auction for a virtual item that gets displayed on the front end of the forum.
• - Members can make a bid on the item. When making the bid, we would only update the entry for this member, or we insert it if the key doesn't exist in the dictionary.

Currently we have to set the data in the key so we can update an entry. There is a chance (this becomes greater on more active forums) that the page contains dirty data (people go AFK then come back and hit "Post"). This dirty data would potentially nullify any data that was updated prior, which is bad.

Now you might be thinking that push solves this issue, but it doesn't. It's only useful if the member isn't in the key data.

Using the current plugin API, the set method would remain unchanged.

When making the request:

• - Server side looks at the type of key.
• - If a dictionary type, then it looks up the key (object_id) in the dictionary.
• - If entry exists, then only this entry gets updated with the value that is passed in from the Plugin API.

This way only that entry gets updated, no more worrying about dirty data. While this doesn't solve all dirty data issues, it helps open up some ideas for plugins some of us have had but put off due to this issue.

The implementation server side is straight forward, and front end basically remains unchanged. I kept it simple like this so there was less work to be done.

It would be nice to extend this further to allow for removing an entry, and clearing the dictionary as well. In fact I would say it would be a requirement.

[Matej]

Hey Peter!

We're actually currently implementing the base plugin JS functionality, ensuring that everything from v5 works in v6 (before adding any new functionality), so the suggestion comes at a very good time!

First off, I love the suggestion and we have already started discussing it internally to see if there's a solution we could implement.

Presented as a super forum (dictionary) key, would it be preferable for this to just be limited to a single key for the entire forum (equivalent to a regular super forum key), or perhaps add the options of a super user dictionary key as well (each user having their own dictionary)?

Thinking about this even more broadly, this also opens the door to something like a thread key dictionary, limiting the dictionary to a single thread, or maybe even a custom page dictionary key. Presumably for something like forum currency and auctions, this would need to be accessible forum-wide, regardless of the presence of threads/posts/users.

I'll be back with any more questions or thoughts that might arise as we discuss this further.

[Peter]

Hi Matej

Thanks for responding.

So my example for auctions would suit a super forum dictionary key, which would allow it to be available anywhere (like the current super forum key).

I would be really happy with just the super forum / user dictionary key.

I think the main 4 would be good, even though the last 2 I couldn't think of a good reason for them having a dictionary type.

• Super forum dictionary key: i.e Auction house.
• Super user dictionary key:  i.e Member auction / shop.  This would allow for an auction / shop be setup by members.
• Thread dictionary key: I can't think of a good use case for that one right now (am trying to think of a current plugin where dirty data is an issue but can't).
• Post dictionary key: Might be useful for plugins that do something like "Thanking" the user in the post.  Though I feel this is acceptable to dirty data as it doesn't cause big issues like an auction would have.

Obviously the more the better, but even just the first 2 opens up a lot of ideas which are floating around in my head right now (and more ideas are flowing in the more I think about this ).

Going to tag a few active plugin devs to see if they have any thoughts: Chris Lynx elli Todge

Thanks

[Lynx]

I love this idea! This sounds like a perfect (at least, MUCH better than what I have now) solution to the Leaderboard issue with the Scavenger Hunt plugin. If only that member's data was updated in the Leaderboard, that would basically eliminate the big differences that some of the more active forums have been reporting in regards to the Leaderboard data compared to the member's personal data due to dirty data when the Leaderboard key gets written to.

I'm not sure if there's a JS or JQuery equivalent, but in BASIC - I could write directly to the Index I wanted to. If there's a JS/JQuery equivalent to being able to do that, that might be a way to implement this. In this case, I think (if possible) that using object_id (such as user's ID, if stored in key data) would be used instead of the actual Index location.

If need be, I still have some of my old BASIC programs that I could look through to see if I can give a BASIC code example of how that would be done - so maybe a JS/JQuery equivalent can be figured out.

[Peter]

Lynx

This would definitely solve your leaderboard issue that some forums are having with your plugin.

There is nothing really to do on the JS side from our point of view, as the dictionary (Perl Hashes) would be handled on the server. When you attempt to get the value of your key, then it will be an object with key value pairs.

• This key could be anything you like, but it needs to be unique.  
• The value that gets set can be whatever you like, as it will be passed to the backend as a string (JSON.stringify).

pb.plugin.key("my_super_f_dict_key").add({ object_id: key, value: val }) // creates the entry
pb.plugin.key("my_super_f_dict_key").update({ object_id: key, value: val }) // updates the entry
pb.plugin.key("my_super_f_dict_key").clear() // clear dictionary
pb.plugin.key("my_super_f_dict_key").remove(object_id) // remove entry

object_id is used in keeping with the current property naming used. Ideally it would be called key, but it doesn't matter too much.

A simplified version of the API could just stick with set, however I see reason to have inserting and updating of dictionary entries as separate API calls.

update should fail if the key doesn't exist.

Standard callbacks (promises (then, catch) please) should be added for success and failure. I may want to know that an update failed.

[elli]

Anything that makes data easier to work with is a big plus, imo.

I'm kind of wondering why keys don't behave like this already?

[Lynx]

Aug 9, 2019 14:54:58 GMT -8 Peter said:

pb.plugin.key("my_super_f_dict_key").add({ object_id: key, value: val }) // creates the entry
pb.plugin.key("my_super_f_dict_key").update({ object_id: key, value: val }) // updates the entry
pb.plugin.key("my_super_f_dict_key").clear() // clear dictionary
pb.plugin.key("my_super_f_dict_key").remove(object_id) // remove entry

object_id is used in keeping with the current property naming used. Ideally it would be called key, but it doesn't matter too much.

Ah, maybe I'm not understanding then because I was picturing something like:

pb.plugin.key("my_super_f_dict_key").add({ object_id: userID, value: newValue })

where userID is taken from pb.data('user').id and newValue would be from the user's current count plus the count just collected. I guess I'm confused where you state above that object_id would ideally be called key.

Aug 10, 2019 10:52:29 GMT -8 elli said:

I'm kind of wondering why keys don't behave like this already?

Remember that plugins were new to V5 as well, and the Devs were not sure how resource intensive they may have been. My guess would be that they wanted to keep it somewhat minimized while still having the necessary functioning (like set or set_on) until they knew how much resources plugins would take. Now that they know, I can see it quite possible that the keys may get better control (as per Peter's suggestion) - among other things.

[Peter]

Aug 10, 2019 22:25:34 GMT -8 Lynx said:

Ah, maybe I'm not understanding then because I was picturing something like:

pb.plugin.key("my_super_f_dict_key").add({ object_id: userID, value: newValue })

where userID is taken from pb.data('user').id and newValue would be from the user's current count plus the count just collected. I guess I'm confused where you state above that object_id would ideally be called key.

The naming of the property that would be for the key (in this case it's object_id in the examples) isn't important. For your example that would work, as the key in the dictionary would be the user ID. As long as it's unique it will be fine.

So an example of your dictionary data would be like this for your plugin. The key being the user ID and the value being the count.

{

	45: 5,
	9: 262,
	1682: 44
	88: 11

}

Instead of updating the whole key, a dictionary key would allow you to update only a specific entry without altering the whole key.

Quick example:

let user_id = pb.data("user").id; // 88
let data = pb.plugin.key("scavenger_lb").get(); // Gets all dictionary data from key

// This is how I would like the API to work, separate calls for actions
// performed on the dictionary.  Yes, we could go with 1 call, but there
// may be cases where you don't want to do anything if an entry doesn't exist
// and you want to handle that via the callback.

if(data[user_id]){
	pb.plugin.key("scavenger_lb").update({

		key: user_id,
		value: (data[user_id] + 1) // 12
	
	}).then(r => console.log("Dictionary Updated")).catch(r => console.log(r.error));
} else {
	pb.plugin.key("scavenger_lb").add({

		key: user_id,
		value: 1
	
	}).then(r => console.log("New entry added")).catch(r => console.log(r.error));
}

There is still a chance of dirty data, but it can only effect the current user's entry. We can also solve this issue as well by syncing data between open tabs / windows for that user.

So with this type of key it's a game changer as it opens up new ideas.

[adminabp]

This would be a BIG help with a plugin like the Recently Updated Threads Sidebar by Todge.

[Todge]

Excellent idea Peter

A plugin of this type would certainly help with the Recently Updated Threads plugin, but would cause issues too. Right now all the key data is stored in an array, all I need to do is keep that array in the order it is to be displayed, and keep it at the length specified by the user. Using a key of this type, the plugin would have to keep track of when each element was updated and to actively remove those elements that are no longer needed... That could mean several calls to the servers to remove elements at the same time, though it could be managed.

Peter , couldn't the add, update and remove functions all be done with the set/set_on function? If the object doesn't exist then it will be added, if it does it will be updated, and if the value is blank it will be removed, only the clear function needs to be separate.

[Peter]

Todge

As already mentioned in my posts, yes you could, however it reduces the flexibility of this type of key and opens up to dirty data being set. There could be cases where you don't want a key to be updated if it exists, or inserted if it doesn't exist.

Now you may be thinking... but we have the data from the dictionary key on the page, so we would know if the key exists or not. This fine until we call the API, remember we can't get fresh data back, so with a plugin where the user isn't the main updater of the key, then dirty data could still be set, this is why having options / separate calls helps as the server will respect what we really are wanting to do. If the entry is removed from the key (i.e member of staff removed a users entry from the dictionary), but the user sits on the page getting ready to set, then we have a problem with dirty data again. With calling update, we are telling the API to only update the entry if it exists, and this is checked on the server. By doing this, we minimize dirty data which is the goal of this type of key.

[Todge]

Peter

Yes, I understand that. I was thinking that if member A does something that sends an 'add' request, an element is created.. All well and good, but if member B is already on the forum, we wouldn't know that the element has already been created, and so would send another add request, causing the data to be overwritten or ignored, depending on how the key would function. If 'set' was used in both cases then the server would decide either to add or update, depending on whether the element existed or not.

Either way, I love the idea as it means we could update parts of the key data without potentially losing data from other parts.

[Peter]

Todge

No, the overwrite would not happen.

In your case of Member A and B; When B calls add, then it would fail because it was already created. This would be relayed back to you via a callback, then it's up to you what you do. In your case you may want to then issue an update, or gracefully bail out and let the user know. This solves dirty data issues.

It's better for them to be separate methods and callbacks with good error info so we know what to do. It's better to do it this way to minimise dirty data. The server should be doing as much as possible to help us out and return good status codes / info so we can decide on what action to take.

Generally in most languages, a dictionary / hash table just has an add (or equivalent) method for inserting / updating and would either overwrite or throw an error if the key exists. In this case we need to deviate from that by adding the second method because of our issue.

Whatever method that is decided on will be a ton better than what we have today, but we should try and get it exactly how we want it right from the start.

[Todge]

Aug 12, 2019 16:22:32 GMT -8 Peter said:

...Whatever method that is decided on will be a ton better than what we have today, but we should try and get it exactly how we want it right from the start.

That's for sure.

With any luck we will get exactly as you have suggested.