String strip

[Resike]

Calling this function on strings which containts "ê" character will broke other string functions like string.len and they won't return anything.

print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+()]+", ""), string.len(x))

Anyone know a hack to make this work properly?

[Choonstertwo]

You could use a slightly less restrictive pattern that matches everything after the first dash:

("PLAYERNAME-Aggra(Português)"):gsub("%-.+", "")

You could also use the WoW-specific strsplit function:

(strsplit("-", "PLAYERNAME-Aggra(Português)", 2))

Note the extra set of parentheses to discard all return values except the first.

[Phanx]

@Choonster: There's nothing wrong with the gsub part. The problem is that Lua's string.len is not UTF8-aware. Blizzard does provide a strlenutf8 function that is UTF8-aware, however.

But the given example code should not work anyway, unless you've defined x somewhere else:

print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+()]+", ""), string.len(x))

[Banknorris]

Originally Posted by Phanx There's nothing wrong with the gsub part.

The result of the gsub part is

PLAYERNAMEês)

Looks broken to me.
%a does not account for ê, only e or E (and other letters of course).

[Resike]

Originally Posted by Banknorris The result of the gsub part is PLAYERNAMEês) Looks broken to me. %a does not account for ê, only e or E (and other letters of course).

It's stips the string properly for me, but i actually used the "%-[%a+é'()]+" pattern in my real function.

[Resike]

Originally Posted by Phanx @Choonster: There's nothing wrong with the gsub part. The problem is that Lua's string.len is not UTF8-aware. Blizzard does provide a strlenutf8 function that is UTF8-aware, however. But the given example code should not work anyway, unless you've defined x somewhere else: print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+()]+", ""), string.len(x))

Yeah my bad:

print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", ""), string.len(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", "")))

I think the string library should be always UTF8-ready. The Blizzard strxxx ones are not tho.

[Resike]

Originally Posted by Choonstertwo You could use a slightly less restrictive pattern that matches everything after the first dash: ("PLAYERNAME-Aggra(Português)"):gsub("%-.+", "") You could also use the WoW-specific strsplit function: (strsplit("-", "PLAYERNAME-Aggra(Português)", 2)) Note the extra set of parentheses to discard all return values except the first.

The reason i used the %a+ pattern because sometime i also have to handle string like this:

"PetName-ServerName <OwnerName-ServerName>"

I havn't tried the built in strsplit but i don't think it would make a difference, usually the default Blizzard functions handle utf8 stuff even worse.

[Resike]

I managed to make it work with a pattern like this:

print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+êé'()]+", ""), string.len(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+êé'()]+", "")))

Returns: "PLAYERNAME", 10

The weird part is this function strips properly too, however it brokes the string.len:

print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", ""), string.len(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", "")))

Returns: "PLAYERNAME"

And i only used the "é" pattern here to properly handle some French server names like: "Chants éternels".

[Choonstertwo]

Originally Posted by Resike The weird part is this function strips properly too, however it brokes the string.len: print(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", ""), string.len(string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", ""))) Returns: "PLAYERNAME" And i only used the "é" pattern here to properly handle some French server names like: "Chants éternels".

This isn't actually stripping properly, it's just broken in such a way that it looks like it's working.

é and ê are both two-byte characters that share their first byte. When you use é in the pattern, you're actually using \195\169. Since Lua's string functions operate on bytes rather than UTF-8 characters, the first byte (195) matches the first byte of ê (\195\170) and leaves behind the second byte (170), which is invalid by itself. When WoW's print function encounters this invalid byte, it simply ignores anything after it.

This snippet escapes any bytes > 127 (the end of the ASCII-compatible section of UTF-8):

local stripped = string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", "")
local escaped = stripped:gsub(".", function(c) local b=c:byte()if b > 127 then return "\\" .. b end end)
print(escaped, #stripped) -- Output: PLAYERNAME\170s) 13

[Lombra]

Why not "%-[^ ]+"?

[Resike]

Originally Posted by Choonstertwo This isn't actually stripping properly, it's just broken in such a way that it looks like it's working. é and ê are both two-byte characters that share their first byte. When you use é in the pattern, you're actually using \195\169. Since Lua's string functions operate on bytes rather than UTF-8 characters, the first byte (195) matches the first byte of ê (\195\170) and leaves behind the second byte (170), which is invalid by itself. When WoW's print function encounters this invalid byte, it simply ignores anything after it. This snippet escapes any bytes > 127 (the end of the ASCII-compatible section of UTF-8): local stripped = string.gsub("PLAYERNAME-Aggra(Português)", "%-[%a+é'()]+", "") local escaped = stripped:gsub(".", function(c) local b=c:byte()if b > 127 then return "\\" .. b end end) print(escaped, #stripped) -- Output: PLAYERNAME\170s) 13

I guessed something like this would be in the background, i guess there is noting to do about it, since thats how gsub should work.

It's just a bad method to escape similar characters with the same starting byte, at least for this case.

[Resike]

Originally Posted by Lombra Why not "%-[^ ]+"?

It would strip down my ending ">" too.

[Lombra]

Originally Posted by Resike It would strip down my ending ">" too.

So you need to process the following:

PlayerName-ServerName
PetName-ServerName <OwnerName-ServerName>

and the result in either case should be what exactly?

[Banknorris]

local function name_split(fullname)
    local name,server,ownername
    local ownerstring = fullname:match(" <(.+)>$")
    if ownerstring then
        name,server = fullname:match("([^-]+)-?(.*) <")
        ownername = ownerstring:match("[^-]+")
    else
        name,server = fullname:match("([^-]+)-?(.*)")
    end
    return name,server,ownername
end

[Voyager]

Originally Posted by Resike It would strip down my ending ">" too.

Try this one:

    local name = "PêtName-Aggra'mar(Português) <Ownêrname-Aggra'mar(Português)>"
    if strfind(name, " <") then
        name = gsub(name, "%-[^ ]+([ >])", "%1")
    else
        name = gsub(name, "%-[^ ]+", "")
    end

It returns "PêtName <Ownêrname>" for me.

[Resike]

Originally Posted by Lombra So you need to process the following: PlayerName-ServerName PetName-ServerName <OwnerName-ServerName> and the result in either case should be what exactly?

PlayerName
PetName <OwnerName>

There are 4 special server names which could interfere mostly:

-Azjol-Nerub
-Aggra(Português)
-Blade's Edge
-Marécage de Zangar

[Resike]

Originally Posted by Voyager Try this one: local name = "PêtName-Aggra'mar(Português) <Ownêrname-Aggra'mar(Português)>" if strfind(name, " <") then name = gsub(name, "%-[^ ]+([ >])", "%1") else name = gsub(name, "%-[^ ]+", "") end It returns "PêtName <Ownêrname>" for me.

I think this could work also, i'm just not sure it's faster then the single gsub call or not. Need to test it.

[Voyager]

Originally Posted by Resike I think this could work also, i'm just not sure it's faster then the single gsub call or not. Need to test it.

Seems like this could do the trick:

gsub(name, "%-[^ >]+", "")

Upvaluing gsub instead of using string.gsub or name:gsub is also faster.

[Banknorris]

gsub(name,"%-[^ >]+","")

Very nice! What is the purpose of the % in this pattern? I tried without it "-[^ >]+" and it apparently still works.

[Resike]

Originally Posted by Voyager Seems like this could do the trick: gsub(name, "%-[^ >]+", "") Upvaluing gsub instead of using string.gsub or name:gsub is also faster.

Yeah this look nice, in the previous one the extra strfind call was the downside.