Hedir Review : Concerns

baggeroli · Posted: Fri Jan 27, 2006 1:09 pm Post subject: Concerns

First of all, I'd like to say (again) that I really like the HEDIR concept: building a true quality directory based on transparant reviews by volunteers from all over the world. In fact, I think it's the ultimate way to construct a directory. But I think we're facing some major problems. This is not meant to critisize someone personally. I'm honestly concerned about HEDIR and the way things are going.

My biggest concern is still the lack of reviewers. Without reviewers, this directory is on his way to sink into oblivion. Let's take a look at the current status: an estimate of 20-25 reviews per day (reviews, not messages) and the number of steady reviewers is approx. 15-20 (I'm being optimistic here). The number of reviewers has hardly improved over the last couple of months.
Now let's see what measures have been taken lately:

the mandatory 10-reviews-before-inclusion rule was taken out. I found out when I was checking for submission guidelines. I never knew this was changed. In fact, if you take a look at the wiki in the HEDIR Next section, you'll see that Lakhya states "we do not have a consensus on this". My question: if there's no consensus about this, why change it before hand? I absolutely do not agree with taking this rule out.
the short explanation about HEDIR inclusion policy, which was always presented at the submission form, is now hidden in an obscure link in the "choose your submission-type" page. No one is ever going to click it IMO. The HEDIR review system should be thrown into submitter's face in a clear and noticable fashion.
A wild idea: why don't we explain the HEDIR system before submitter gets to the submission form. To prevent that submitter immediately clicks "next" without reading the explanation, we could force the submitter to read it by providing a form beneath the explanation with the following statement: Please copy and paste the line "You have to review 10 sites to be included in the directory" in this form to confirm that you've read HEDIR's policy and click next. Make it as obvious as possible.

-----------------------------------------------

My second concern is about the reviewing system itself. This problem can be divided into two sub-problems: the lack of an ordering system in the Review forum and the mixed voting/reviewing system.

Ordering System:

This was discussed here. The forum still isn't ordered. There is a priority list at the top-right "Today's sites for review" but it only gives 5 sites without any further detail and it never works for me. I never get any sites.
There is a queue list which does have an ordering system (btw: I have to log out to get the link), why can't this be implemented in the Review forum? Is it a speed thing? If so, maybe it's possible set a up cron job every hour or so to refresh the queue list and take the first 5 pages only for the review forum. Leave the rest of the forum unsorted.

Mixed voting/reviewing system:

What's the premier feature that distincts HEDIR from other directories? Transparant reviewing of sites by members which are openly responsible for their reviews! This is the whole basis of HEDIR. Without it, it would be just another directory: editors decide which sites are included. No one knows why a site has been approved or rejected.

I'm under the impression that the voting is getting more and more importance than the reviewing and I disagree with that. Just take a look at HEDIR Next and you'll see that positive reviews aren't counted at all anymore. In fact, you're only required to have a total of 10 reviews. Whether they're positive or negative doesn't matter. Only the votes count: you need 75% or more. This is truly ridiculous. Why on earth should we review then?

I think the voting undermines the whole principle of HEDIR. It's anonymous (even guests can vote), not accountable and uncontrollable. Therefore it's meaningless compared to argumented reviews. I understand that it's easier for automation reasons, the code can do all the hard work, but it takes away the uniqueness of HEDIR. That's why I don't like it and I'm not even speaking of the remote voting. IMO that's asking for trouble and it will be the death of HEDIR (if you can find anyone who'll actually put the script on his/her site). I understand HEDIR needs back-links but this is not the way. Let quality and time do their work.

I suggest an integration of the voting and reviewing. No more reviewing and voting. The review becomes the vote. The vote can only be casted together with a review. That way you have the advantages of both systems: you have a argumented, accountable review and you have a vote to work with code-wise.

-----------------------------------------------

My third concern is about the priorities. Take a look at the priority list at HEDIR Next. As I've stated before: the true power of HEDIR lies in it's reviewing system and therefore the Reviewers are the most important people over here. All 3 projects with high priority and the one completed project are projects which do not concern the Reviewers. Projects that do, have a low or an undetermined priority (projects 4, 7, 8, 13, 15, 16 and 17). An ordering system in the Review forum isn't even mentioned! This has to change IMO.

Again, my intentions are not to critsize or offend anyone. If I did, I apologize. I'm just a HEDIR addict who believes in the concept but thinks we're going the wrong way.

Edmond Dantès · Posted: Fri Jan 27, 2006 2:09 pm Post subject:

GOOD!

Google Junky · SuperMOD Joined: 01 Jan 2006 Posts: 3868 286 Hedir Points

Congrats baggeroli!
Very well written

I recommend baggeroli for HEDir <-- is that a vote?

The horse before the cart... We need to make sure we put this into perspective as to what should happen before the other. More reviewers verses getting all the crap in HEDir fixed before more people show up. Doing both at once isn't feasible IMO. Fix what ails HEDir and its current members. Then go after growing more. When you start fixing things and get into errors then bringing new people will surly run them off when the errors pop up.
Just my thought though.

I have many many issues with the system of ordering. I could review more sites and maybe make up for the some of the absence of needing more reviewers if I had some sort of reviewing order to do this with. I kept think that if I had a downloadable csv, xls, or some type of tab-delimited file of sites needing reviewed then I could go after them one by one. I'd rather not review a site for a person that hasn't made an attempt to review anything themselves. Again "driveby posters", who will never be back and could care less to look back and see how their review went.

I don't plan on leaving HEDir. We just need to get a few things fixed to make this an even better experience. The important issues have been well known for awhile. Its now time to act on those issues.

Google Junky

redhex · Joined: 10 Nov 2005 Posts: 559 18636 Hedir Points

I second baggeroli for his suggestions. Lets get our priority striaght and hedir will grow.

Feydakin · Posted: Fri Jan 27, 2006 3:41 pm Post subject:

I do agree with the issue of ordering.. I know that I spend a lot of time in the bottom of the stack trying to pull out the really good sites (and occasionally the bad ones we need to dump) but that is usually overshadowed by the fact that most of those people rarely, if ever, return..

Personally, I was, and still am, agasint the mandatory review requirement for submitters.. It's a strip of duct tape stuck over a larger problem since it will be very rare that we gain and retain real members of the community.. Most are sticking around, if they do at all, to bang out 10 quick reviews and move on..

I think that the biggest issue at this point is organization of sites to be reviewed.. It's very hard to keep track of what you have reviewed, what needs to be reviewed, and things that you are watching for improvement before voting one way or the other.. I know that I spend far too much time reading older threads to see if I had voted or not and where they are in the vote process..

But, like many here, I do plan to be here for the long haul and hope that we can make this a great directory..

BTW, we've been having some serious family issues the last few weeks (we are averaging a death a week since the year started) and I am spending far less time online than I used to.. I hope that changes soon..

Steve -

Google Junky · SuperMOD Joined: 01 Jan 2006 Posts: 3868 286 Hedir Points

baggeroli · Posted: Fri Jan 27, 2006 4:00 pm Post subject:

Feydakin · Posted: Fri Jan 27, 2006 4:08 pm Post subject:

It's like we need three flags we can add to the reviews.. Need reviewed, been reviewed, and watching and then being able to simply (hehe I said simply in a programming suggestion) sort based on that..

I understand the use of phpBB for this process, but it's really the wrong tool I think.. At least in it's present form..

baggeroli · Posted: Fri Jan 27, 2006 4:28 pm Post subject:

Well, the "need reviewed" and the "been reviewed" are already logged by the votes. Maybe we need a third option "undecided" which would be the threads to watch.

There's also a need for withdrawing votes or changing votes but I guess that would get a bit complicated. I assume the vote per username is logged so maybe there is a way to allow to vote another option but to disallow the same option. E.g. you can vote no if you voted yes before, but you can't vote yes again.

Feydakin · Posted: Fri Jan 27, 2006 4:36 pm Post subject:

Yes, they are logged by votes, but if I don't show up for a day or two I will forget what I had already voted on and wind up clicking on lots of threads that are cluttering up the space that I'd rather not see..

In an ideal world once I vote on something it goes away and I never have to look at it again.. Then if I don't vote and just click a "watch this thread" button it will go into a different area where I can come back to it later without having to hunt for it.. Or getting an eMail alert every time someone posts to the thread.. Or having to bookmark it..

This leaves us with 'everything else'..

I admit it, I'm lazy.. I have far too much work to do to spend a whole lot of time trying to keep organized with reviews on my own..

Steve -

baggeroli · Posted: Fri Jan 27, 2006 4:55 pm Post subject:

Hehe, I know Steve. I just meant that since the votes are already logged, it would be possible to flag each submission according to the user's logged votes. I have the same problem as you: I constantly forget which site I've reviewed and which not, let alone which site I have to watch.

Davenut · Joined: 05 Jan 2006 Posts: 93 3011 Hedir Points

I have a concern about this aspect of HEDIR:

baggeroli · Posted: Fri Jan 27, 2006 6:02 pm Post subject:

I completely agree with you, Davenut. That's really not the way to go. Thanks for the heads-up, must have over read that.

winterfrost · Posted: Fri Jan 27, 2006 11:08 pm Post subject:

baggeroli, you know you have my 100% backing on all of this. As you also know, I've pretty much given up on reviewing because there has been NO progress in any of these areas for the... oh... 3 months or so that we've been discussing them.

I used to be Hedir's #1 Advocate until it became clear that the development (that is development in both the "programming" and "community building" senses) priorities are completely askew. It seems to me that the owners are worried about the paint job and upgrading the stereo system... even though the transmission is shot.

Edmond Dantès · Posted: Sat Jan 28, 2006 8:43 am Post subject:

Lakhya? Anthony? Norah?

We are here!!!