Mods Rating System on WoWI

[Steil]

As a User and not a coder, the favorite list should be the determining factor. People can account hop all they want, but the numbers of ppl having it as a favorite addon would be how you decide the popularity vote. Not a 1 out of 100,000 vote just a flat how many ppl have it favorited. If you must have a diminishing timeframe that the votes count. Make it so that the votes only count ppl that have logged in and checked their favorite list in the past year or past 3 patch dates.

[Taffu]

The "Reviewer Panel" idea I think is a good one, and people should know that it's mutually beneficial to both the author(s) and the user(s).

For the Users, it can offer an "At-a-glance" overview of the AddOns strong points & weak points (much like the average game review does) giving users an idea if it fits the bill for what they're looking for. Some users look for performance over ease of configuration, and vice versa. Much like the difference between, say, LynStats and SLDT...a user that prefers extremely lightweight data representation and doesn't mind tinkering with LUA will take LynStats, and those that like ease of configuration at the cost of a little bit of performance will take SLDT. At-a-glance info can pass information like this along, say with a badge system that awards things like "Easy to configure" or "Some LUA knowledge required". Or even categorizes ratings associated to key points of the AddOn.

For the author's, it can provide another developer/author insight into their AddOn. This can help them not only better their AddOn(s), but help them understand something they might not have known about in their initial design. There are all levels of authors from those doing the Copy/Paste/Modify thing to those creating absolute, 100% original & technically complex AddOns that break a barrier not yet passed. We can all learn something from someone else, and there's almost always something we didn't know about before. Opening the lines of communication through thorough AddOn review for author's could be extremely beneficial to the general quality of AddOns site-wide.

As far as concept goes, it could be something as simple as the review panel thoroughly takes a look at the source, functionality, configuration, and more to provide something like a "3-point inspection" that awards ratings such as:

Configuration: A base to descibe to the user(s) how easy it is to setup. Ratings like "Some LUA knowledge required" or "Minimal configuration" could describe quickly what a user can expect on first load.
Performance: A base to describe some general idea of how performance-friendly the AddOn is. Ratings like "< 100kb 'Efficient'" or "> 2mb 'Heavy'" could give in-game usage as well as load-performance ideas.
Functionality: A base to give the user an idea of how well the AddOn does "what it does", while giving feedback to the author as to how well they've built the AddOn "to" do "what it does".
Overall Grade: I'm thinking a grade scale like "A" through "F". With + and - of course. :P

The more I think about it, the more ideal a system "similar" to this sounds to be. Mutually beneficial, effective, offers at-a-glance ratings as well as some general comments that can be done by an "editorial" perspective from within the review panel based on individual reviewer feedback. Is there a way to pitch something like this to the Mod's for consideration (you guys still reading?)

[Bouvi]

To be honest I never rely on a rating system, downloads, etc. I read the comments if there are any and the Mod description. Comments are the best way to determine if it is something I want.

I personally don't care if 1,000,000 people or just 10 downloaded it. Out of the 1,000,000 downloads maybe 999,999 found it did not work for them so in reality only 1 person is using it.

I never did like rating systems for the very reason stated in Cairenn's original post. If you are going to have one, I would only allow it when comments were made. Or just leave it out.

I check the new/updated addons via RSS on my iPhone so I keep an idea of what is out there and check out anything that sounds interesting. Again comments and the mod description are what I look at.

[erica647]

I just want to state my opinion as far as a "favorites based" rating system goes... I don't think that's really a good system as a lot of mods are targeted to a certain group of players. For example, a lot of the minimalist mods do a simple thing and most do it well. Even though there may not be a lot of downloads and may not be on a lot of people's favorites list, that does not really mean it's a bad mod. On the other hand, a mod that does a lot of things and is really easy to setup may have a lot of "bloat" (bells and whistles, high memory and cpu usage cycles, etc.) may have a high amount of favorites since it's easy to setup but again doesn't necessarily mean it's a good mod or well written. A lot of this just depends on what the "average" player is really looking for in a mod.

Another thing about a "favorites based" system that I find a bit troubling is that I think it may keep some mod authors from even updating their mods if they think it's too simple and targeted to certain players for fear of a bad rating... since it probably won't have a lot of favorites. If we do decide to go with some type of "favorites based" system, let's at least try to split the mods up into categories so that the number of favorites is weighted to reflect "targeted-audience" mods versus "do-everything" mods aimed at the masses.

[Duugu]

A quick compilation of all statements/ideas/whatever I've read in this thread so far.

Basic findings

A simple (eg. "five star") rating don't pinpoints the addons quality. Instead it shows how popular an addon is.
Authors would like to have a system that shows how popular the addon is and that identifies improvement opportunities.
Users would like to have a system that shows the "best fit" addon for a specific purpose.

Proposals/ideas

• Ratings
  
  • "+" or "-" ratings only
    
  • weighted ratings by
    • number of post the user wrote
      
    • number/type of ratings the user has done
      
    • downloads the user has done
  • positive ratings only
    
  • rating are reset periodically (every month/quarter/year/...)
    
  • comments
    • comment is mandatory for the first/second download
      
    • additional "Was this comment helpful to you" system
      
    • "Pro & Con" rating system without numbers
  • multiple ratings with several categories
    
  • rating for certain users (staff/authors/...) only
    
  • separate ratings for every addon version
    
  • "decreasing over time" ratings
    
  • private "user-to-author" rating system
• Recommendations
  • "Would you recommend this addon to others?"
    • "yes" and "recommend mod X instead",
      
    • additional "Was this comment helpful to you" system
• Badges/tags
  • examples: "staff picked", "was mod of the month", "Quality Code", "Simple and Clean", "Must Have", "Best in class"
    
  • given by staff/authors/a regular group/all users
    
  • one badge or multiple/unlimited badges per user
• Review
  • by a regular group of authors/staff/...)
    
  • by users (eg. Amazon)
    • additional "Was this comment helpful to you" system
  • for every new/updated addon
    
  • includes:
    • short description
      
    • Rating for specific categories (eg. ease of use, simplicity, design, concept, ...)
      
    • overall score
      
    • "more addons like this"
• Favorites only
• Downloads only
• Overview who's using/downloading which addon
• Addon updater phones home to generate popularity statistics
• More/new addon categories ("show me more addons like this" system)

[eiszeit]

Originally Posted by p3lim Having Top Favorites, Top Downloads, the spotlight and the newest updates/uploads pretty much covers it, I can not see any need for a rating system, 'cause the favorites pretty much covers that, in a cleaner way (you dont have 'vote my addon!' crap with favorites) my 2 cent

What he said.

I also like what Taffu said. Something like a "Badge" system where the author can mark some points like "easy to configure" etc..

[Seerah]

Originally Posted by Duugu A quick compilation of all statements/ideas/whatever I've read in this thread so far.

Thank you for that, Duugu.