Skip to content

Memory cache + Biome API #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
leucome wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Memory cache + Biome API #6

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 30 additions & 20 deletions init.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@

]]--

local wp = minetest.get_worldpath() .. "/luscious"
minetest.mkdir(wp)
gt = {}
tablesize = 0
Copy link
Copy Markdown
Owner

@sofar sofar Jan 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should both be local since they are only used inside this file.


local mgp = minetest.get_mapgen_params()
local chunksize = 16 * mgp.chunksize
Expand All @@ -28,25 +28,33 @@ local function cmpy(p2, y)
return math.max(1, p2)
end
end

local function chose_color(pos)
-- Calculate color from biome heat and humidity.
local biome_data = minetest.get_biome_data(pos)
local hum1 = biome_data.humidity
local temp1 = biome_data.heat
local vh = math.floor(math.min(math.max(math.floor(hum1), 0), 100) / 6.6)
local vt = math.floor(math.min(math.max(math.floor(temp1), 0), 100) / 6.6)
vt = vt - math.max(math.min(vt, math.floor(pos.y / 16)), 0)
return vh * 16 + vt
end

local function on_construct(pos)
-- get chunk from pos
local v = vector.apply(pos, function(a) return math.floor((a - 48) / chunksize) end)
local o = vector.subtract(pos, vector.apply(v, function(a) return (a * chunksize) + 48 end))
local l = o.z * (chunksize) + o.x
local p = minetest.hash_node_position(v)

local f = io.open(wp .. "/" .. string.format("%d", p), "r")
if not f then
minetest.log("error", "unable to find map for " .. string.format("%d", p))
return
end

local z = f:read("*a")
f:close()
local map = minetest.decompress(z)

local map = gt[p]
local mv = 0
Copy link
Copy Markdown
Owner

@sofar sofar Jan 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 0 value here is never used. Just use local mv here. Luacheck will spot this and warn about it otherwise.

local node = minetest.get_node(pos)
node.param2 = cmpy(string.byte(map, l + 1), pos.y)
if map == nil then
Copy link
Copy Markdown
Owner

@sofar sofar Jan 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if not map then

mv = chose_color(pos)
else
mv = cmpy(string.byte(map, l + 1), pos.y)
end
node.param2 = mv
minetest.swap_node(pos, node)
end

Expand Down Expand Up @@ -146,12 +154,14 @@ minetest.register_on_generated(function(minp, maxp, blockseed)
h2 = math.floor(math.min(math.max(math.floor(h2), 0), 100) / 6.6)
map = map .. string.char(h1 + (h2 * 16))
end
local p = string.format("%d", minetest.hash_node_position(v))

local f = assert(io.open(wp .. "/" .. p, "w"), wp .. "/" .. p)
f:write(minetest.compress(map))
f:close()

local p = minetest.hash_node_position(v)
if tablesize > 75 then
gt = {}
collectgarbage()
Copy link
Copy Markdown
Owner

@sofar sofar Jan 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely not so bad, but, I think you can avoid calling collectgarbage() here entirely. I also think that this method of wiping all cache is relatively inefficient, as you're constantly redoing the caches.

A not-so-complex approach is to keep hashing on p, but not embed only the data in there, but also a timestamp and delete only old things from the cache.

so gt[p].timestamp and gt[p].value.

Then you can walk the array, and erase older values than some limit or threshold.

Perhaps you could even embed a cache hit counter in the record and use that to eliminate things that get none or little cache hits, and always keep items with lots of cache hits.

When you do that, you want to make sure you don't remove just 1 item and have to re-prune when 1 item is added, just let the cache grow e.g. to 128, and then prune down to 64.

I would avoid calling collectgarbage() if you can at all times and let the lua VM take care of this, if you can.

Copy link
Copy Markdown
Author

@leucome leucome Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this part can be improved.
The current code downside is that when the cache is discarded there is a chance to see the biome API called to finish the generation. So about 1 or 2 map every 75 is likely to fall back on the biome API instead.

These biome API calls could be reduce to nearly zero by pruning half the oldest cache instead. Also removing the collectgarbage() would be nice.

tablesize = 0
end
tablesize = tablesize + 1
Copy link
Copy Markdown
Owner

@sofar sofar Jan 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just use #gt to refer to the table size here?

Copy link
Copy Markdown
Owner

@sofar sofar Jan 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine, though, to track size manually.

gt[p] = map
local vm, emin, emax = minetest.get_mapgen_object("voxelmanip")
local area = VoxelArea:new{MinEdge = emin, MaxEdge = emax}
local data = vm:get_data()
Expand Down