Using Data Buckets
An explanation of the use of Data Buckets in EQEmu.

What are Data Buckets?

    Data buckets are a replacement to the well-known qglobals, but they are far more performant, reliable and simpler to use
    You can use data buckets to store values unique to anything you would like, for example
      NPC based flags
      Zone based flags
      Character based flags
      etc.

Functions

Data buckets exist in these 3 main functions in both Perl and LUA
1
get_data(std::string bucket_key)
2
set_data(std::string bucket_key, std::string bucket_value, std::string expires_in)
3
delete_data(std::string bucket_key)
Copied!

Storage

    Data buckets are stored in the [[data_buckets]] table and has a very simple structure
      id
      key
      value
      expires
    Expired data bucket rows will not be queryable via the quest API, they may exist in a table until 5-10 minutes have past and the server will garbage collect and wipe the table clean of expired buckets

Usage Examples

Perl

In this simple example you can see that we are keying by character id to set the flag to make this unique per player, we are setting it with the value of 70, and since we didn't set the optional value of unix time it never expires
1
if ($text =~/character-flag-test/i) {
2
$key = $client->CharacterID() . "-some-epic-flag";
3
quest::set_data($key, 70);
4
quest::say("You have traveled far! You have a mighty (" . quest::get_data($key) . ") epic points!");
5
}
Copied!

LUA

In this example below you can see that we set a simple global flag, we immediately access it, delete it and then try to unsuccessfully access it again because the bucket data had already been deleted
1
if (e.message:findi("test")) then
2
e.self:Say("This is a test!")
3
eq.set_data("lua_test_key", "lua_value");
4
e.self:Say("We just set some bucket data with '".. eq.get_data("lua_test_key") .. "'");
5
eq.delete_data("lua_test_key");
6
e.self:Say("I'm going to try and access the value again... '".. eq.get_data("lua_test_key") .. "'");
7
end
Copied!

Ways to Key Buckets

Keying is simply a way to uniquely identify a flag, if you want to make some data unique to a player, then you would need something to key uniquely to that player, such as their character_id. If you wanted to set a flag uniquely for a NPC for example, you could use the npc_type_id, for a zone you could use the zone_id. All of these circumstances are completely up to you and you have the entire Quest API to grab something that can make something unique!
Some of the examples below should give you some ideas!
By Character
1
$key = $client->CharacterID() . "-some-flag";
2
$value = 70;
3
quest::set_data($key, $value);
Copied!
By Door (And Zone)
1
sub EVENT_CLICKDOOR {
2
quest::say($doorid);
3
if ($doorid == 4) {
4
$key = $zoneid . '-' . $doorid . '-last-person-to-click-door";
5
$value = $client->GetCleanName();
6
7
if (quest::get_data($key)) {
8
$client->Message(15, "You know... the last person to click this door was '" . quest::get_data($key) . "'");
9
}
10
11
quest::set_data($key, $value);
12
}
13
}
Copied!
Result
Database Result
By NPC
1
sub EVENT_DEATH_COMPLETE {
2
$key = $npc->GetNPCTypeID() . "-death-count";
3
quest::set_data($key, quest::get_data($key) + 1);
4
5
$death_count = quest::get_data($key);
6
quest::shout("Man! I've died (" . $death_count . ") times in my lifetime!");
7
}
Copied!
Result
Database

Expiration Examples

    Below in this LUA example we will count the number of times a player has talked to an NPC, first by checking if we've got a bucket set at all, if not we will set an expiration time on it. Each time we call set_data, it will not over-ride the original expiration time unless we pass a new time parameter
1
function event_say(e)
2
if (e.message:findi("hail")) then
3
4
-- Set unique key for the bucket
5
local key = e.other:GetCleanName() .. "_times_talked";
6
7
-- If the bucket is empty, we need to set it
8
-- The first time we will set an expiration on this (86400 seconds)
9
if (eq.get_data(key) == "") then
10
eq.set_data(key, '1', 86400);
11
end
12
13
local times_talked = tonumber(eq.get_data(key));
14
15
e.self:Say("You know... You've talked to me " .. times_talked .. " time(s) today, get a life will ya!");
16
17
-- Increment times talked
18
eq.set_data(key, tostring(times_talked + 1));
19
end
20
21
end
Copied!
Result
Database

Acceptable Time Formats

We have the ability to use time shorthands if need-be, the following are acceptable time inputs
Input
Time Result
15s
15 seconds
s15
15 seconds
60m
60 minutes
7d
7 days
1y
1 year
600
600 seconds

Perl Expiration

    To set an expiration time in Perl, very similarly to the LUA example above, you would simply call your set_data function with an expiration flag as your 3rd parameter like so
1
quest::set_data("my_example_flag", "some_value", 3600); # 3600 seconds = 1 hour (Expire in 1 hour)
Copied!

Benchmarks

    Below are some simple benchmarks used to calculate performance. While even these numbers could be greatly optimized yet, these are plenty good for most use cases that server operators need. If you need even faster temporary data storage within the context of a zone, I would suggest using [[Entity Variables]] as they operate purely in memory
image
1
sub EVENT_SAY {
2
use Time::HiRes;
3
my $start = [ Time::HiRes::gettimeofday() ];
4
5
if ($text =~ /random-write/i) {
6
my $iterations = 1000;
7
my $key_range = 100;
8
quest::debug("Testing random-write... Iterations: (" . plugin::commify($iterations) . ") Key Range: " . $key_range);
9
for ($i = 0; $i < $iterations; $i++) {
10
quest::set_data("key_" . int(rand($key_range)), &generate_random_string(100));
11
}
12
}
13
14
if ($text =~ /sequential-write/i) {
15
my $iterations = 1000;
16
quest::debug("Testing sequential-write... Iterations: (" . plugin::commify($iterations) . ")");
17
for ($i = 0; $i < $iterations; $i++) {
18
quest::set_data("key_" . $i, &generate_random_string(100), 15);
19
}
20
}
21
22
if ($text =~ /sequential-read/i) {
23
my $iterations = 1000;
24
quest::debug("Testing sequential-read... Iterations: (" . plugin::commify($iterations) . ")");
25
for ($i = 0; $i < $iterations; $i++) {
26
$data = quest::get_data("key_" . $i);
27
# if ($data ne "") {
28
# quest::say("Data for $i : $data");
29
# }
30
}
31
}
32
33
if ($text =~ /random-read/i) {
34
my $iterations = 1000;
35
quest::debug("Testing random-read... Iterations: (" . plugin::commify($iterations) . ")");
36
for ($i = 0; $i < $iterations; $i++) {
37
$data = quest::get_data("key_" . int(rand($iterations)));
38
}
39
}
40
41
my $elapsed = Time::HiRes::tv_interval($start);
42
quest::debug("Operation took: " . $elapsed);
43
}
Copied!

Practical Example

In South Qeynos, you will find an NPC named Vicus Nonad. Vicus is a tax collector, but because of his terrible cold, he needs a player to help make the rounds to collect taxes. The early implementation of this quest script utilized quest globals, and below is an example of replacing the quest global functionality with Data Buckets.
Vicus_Nonad.pl
1
sub EVENT_SPAWN {
2
#:: Set a timer "cough" to repeat every 350 seconds (5 min 50 sec)
3
quest::settimer("cough",350);
4
}
5
6
sub EVENT_TIMER {
7
#:: Match the "cough" timer
8
if ($timer eq "cough") {
9
quest::emote("coughs and wheezes.");
10
}
11
}
12
13
sub EVENT_SAY {
14
if ($text=~/hail/i) {
15
quest::say("Greetings, $name. My name is Vicus Nonad. <cough> I am the official tax collector for the fine city of Qeynos. <cough> I serve the will of Antonius Bayle, our glorious leader. <cough> <cough> Please excuse my [" . quest::saylink("cough") . "]. <cough>");
16
}
17
elsif ($text=~/cough/i) {
18
quest::say("Oh, <cough> I am sorry, but it seems I have fallen a bit ill. I was caught out in the rain the other day and my chills have gotten the best of me. <cough> If only someone would [" . quest::saylink("help with today's collections") . "].. <cough>");
19
}
20
elsif ($text=~/help with today's collections/i) {
21
#:: Data bucket to verify quest has been started appropriately
22
$key = $client->CharacterID() . "-tax-collection";
23
#:: Set a data bucket, quest started
24
quest::set_data($key, 1);
25
quest::say("Oh thank <cough> you so <cough> <cough> much <cough>.. Here is the official collection box. Please collect from each merchant on the <cough> [" . quest::saylink("list") . "]. Then bring me back the combined total of all your collections.");
26
#:: Give a 17012 - Tax Collection Box
27
quest::summonitem(17012);
28
}
29
elsif ($text=~/list/i) {
30
quest::say("Oh. <cough> I am sorry.. I forgot to give it to you. Here you go. Be sure to give that back when your job is finished. <cough>");
31
#:: Give a 18009 - List of Debtors
32
quest::summonitem(18009);
33
}
34
}
35
36
sub EVENT_ITEM {
37
#:: Match a 18009 - List of Debtors and 13181 - Full Tax Collection Box
38
if (plugin::takeItems(13181 => 1, 18009 => 1)) {
39
quest::say("<cough> Great! Thank you so much. Here is a small gratuity for a job well done. Thank you again. <cough> Antonius Bayle and the People of Qeynos appreciate all yo have done.");
40
#:: Delete the data bucket
41
$key = $client->CharacterID() . "-tax-collection";
42
quest::delete_data($key);
43
#:: Give a random reward: 13053 - Brass Ring, 10010 - Copper Amulet, 10018 - Hematite, 10017 - Turquoise
44
quest::summonitem(quest::ChooseRandom(13053, 10010, 10018, 10017));
45
#:: Ding!
46
quest::ding();
47
#:: Set factions
48
quest::faction(219,10); #:: + Antonius Bayle
49
quest::faction(262,4); #:: + Guards of Qeynos
50
quest::faction(304,-5); #:: - Ring of Scale
51
quest::faction(273,-10); #:: - Kane Bayle
52
quest::faction(291,10); #:: + Merchants of Qeynos
53
#:: Grant a small amount of experience
54
quest::exp(500);
55
#:: Create a hash for storing cash - 200 to 300cp
56
my %cash = plugin::RandomCash(200,300);
57
#:: Grant a random cash reward
58
quest::givecash($cash{copper},$cash{silver},$cash{gold},$cash{platinum});
59
}
60
#:: Match a 13181 - Full Tax Collection Box
61
elsif (plugin::takeItems(13181 => 1)) {
62
quest::say("Very good <cough> work. But I need both the full tax collection box and the list of debtors. You did get the [" . quest::saylink("list") . "] from me before you left, right? <cough>");
63
#:: Return a 13181 - Full Tax Collection Box
64
quest::summonitem(13181);
65
}
66
#:: Match a 18009 - List of Debtors
67
elsif (plugin::takeItems(18009 => 1)) {
68
quest::say("Very good <cough> work. But I need both the full tax collection box and the list of debtors. You did get the [" . quest::saylink("list") . "] from me before you left, right? <cough>");
69
}
70
#:: Return unused items
71
plugin::returnUnusedItems();
72
}
Copied!
At line 22 above, we see the implementation of the Data Bucket Key. This replaces the use of quest::setglobal. In the Database, we see the corresponding key:
Note that we clean up the key upon handing in the requisite tax money and list with the following (line 41 above):
1
$key = $client->CharacterID() . "-tax-collection";
2
quest::delete_data($key);
Copied!
The original portion of the script for the Quest Global would have been as follows:
1
#:: Match for "help", case insensitive
2
elsif ($text=~/help/i) {
3
quest::say("Oh thank <cough> you so <cough> <cough> much <cough>.. Here is the official collection box. Please collect from each merchant on the <cough> [list]. Then bring me back the combined total of all your collections.");
4
#:: Give a 17012 - Tax Collection Box
5
quest::summonitem(17012);
6
#:: Set the qglobal "tax_collection", to a value of "0", for all NPCs and Zones, and last forever
7
quest::setglobal("tax_collection", 0, 5, "F");
8
}
Copied!
While this implementation may seem to be easier, it should be noted that on a server with many players, running a query through a hash of globals for each event that triggers a look up can cause serious performance issues. Imagine a hash being created for every global qglobal entry (the "5" in the script above), for EVERY event trigger!
Our intrepid adventurer is required to do this quest in order, by speaking with Vicus prior to the merchants in Qeynos who have to pay taxes. To enforce this, we verify that the user has the appropriate data bucket key set (line 9) before offering the dialogue that results in the tax payment:
Mar_Sedder.pl
1
sub EVENT_SAY {
2
if ($text=~/hail/i) {
3
quest::say("Hail, $name. What brings you to the Tin Soldier? We have the finest in previously owned weapons and armor. Feel free to browse.");
4
}
5
elsif ($text=~/tax collection/i) {
6
#:: Data bucket to verify quest has been started appropriately
7
$key = $client->CharacterID() . "-tax-collection";
8
#:: Match if the key exists
9
if (quest::get_data($key)) {
10
quest::say("Here are the taxes, $name. Boy, you call the guards and they take their time to show up but be a few days late on your taxes and they send the goons after you. Sheesh!");
11
#:: Give a 13171 - Sedder's Tax Payment
12
quest::summonitem(13171);
13
#:: Set faction
14
quest::faction(291,-1); #:: - Merchants of Qeynos
15
}
16
}
17
}
18
19
sub EVENT_ITEM {
20
#:: Return unused items
21
plugin::returnUnusedItems();
22
}
Copied!
Last modified 1yr ago