OnUpdate running function and memory saving

[Benalish]

Disabling all add-ons and leaving active only the one created by me, I noticed large swings in memory consumption. This is probably because in OnUpdate event is launched this function, where tables are created all the time:

local function tokenize(C, strng)
    local sChar = string.format('%c', C)
    local sInput = strng or ""
    local tReturn = {}
    for sWord in string.gmatch(sInput, "[^"..sChar.."]+") do
        table.insert(tReturn, tonumber(sWord) or sWord)
    end
    return tReturn
end
 
local function gettok(sInput, iPosition, Separator, iRange)
    local tTokens = tokenize( Separator, sInput )
    if iPosition == 0 then return sInput end
    local iStart, iStop = (iPosition > 0) and iPosition or (#tTokens + iPosition + 1)
    if not iRange or iPosition == iRange then
        return tTokens[iStart]
    end
    if iRange > 0 then
        iStop = iRange
    elseif iRange == 0 or (iStart + iRange) > #tTokens then
        iStop = #tTokens
    else
        iStop = iStart + iRange + 1
    end
    if iStart == iStop then return tTokens[iStart] end
    if iStart > iStop then
        iStart, iStop = iStop, iStart
    end
    return table.concat(tTokens, Separator, iStart, iStop)
end

Where

• sInput = string to manipulate
• iPosition = position of the token inside the string. If lesser than 0, it will be considered the position from the last token to the first. If equal to 0, returns the whole string.
• Separator = ASCII code of the token separator
• iRange = optional: if specified, returns the token from position to range. If equal to 0, return all tokens from position to the end of the string.

Some examples:

local text = "apple.banana.cherry.grape.orange"

gettok(text,1,46)
> apple

gettok(text,-2,46)
> grape

gettok(text,2,46,4)
> banana.cherry.grape

gettok(text,-1,46,-3)
> cherry.grape.orange

Now, can you give me some advice on how to eliminate these fluctuations in memory usage by bringing, as much as possible, some small example, and improve this code?

Thank you in advance

[sylvanaar]

Besides the obvious of not doing too much work in OnUpdate, why don't you reuse the table, since the function is called so much, you will reduce the GC overhead

just place

local tReturn = {}

Right outside of tokenize instead of where it is, then in place of it

wipe(tReturn)

[Benalish]

The best solution of all would resort to the use of metatables. In a Lua language mailing list was suggested this: a table created once and for all for each string

local tok_meta = { __index = { gettok =
   function(tbl,i,j)
     local n=#tbl
     j = j or i
     if i and i<0 then i=n+i+1 end
     if j and j<0 then j=n+j+1 end
   return table.concat(tbl,tbl.sep,i,j) end }}
 
local tokens = function(str,sep)
   local fields={sep=sep}
   str:gsub(("[^%s]+"):format(sep),function(field)
      fields[#fields+1]=field
      end)
   return setmetatable(fields,tok_meta)
end
 
local s = tokens( "apple.banana.cherry.grape.orange",".")
 
> s:gettok(1)
apple
> s:gettok(-2)
grape
> s:gettok(2,4)
banana.cherry.grape
> s:gettok(-3,-1)
cherry.grape.orange

The ideal for me would switch functions gettok and tokens, by inserting tokens inside the tok_meta metatable and gettok as a function of its own, because in addition to it there are other string manipulation functions. But I'm still very little practical with metatable and not would know what can I do, so expect your lights

[Phanx]

Can you explain what you're actually trying to accomplish with this code? Not just "split apple.banana.cherry.grape.orange into cherry.grape" but the general end goal. Metatables are almost certainly a wild overcomplication, but without a higher-level context it's pretty hard to offer any useful suggestions.

(Also, your second post is not only creating a new table, but also a new function, every time you call "tokens" -- those are both things you should avoid at pretty much all costs in an OnUpdate script.)