Hi, I'm Matt Howey. I run a big site called metafilter.com, and this is a copy of my talk
from South by Southwest 2011. I just gave this past Saturday. I was kind of in a location
about half a mile away from the main convention center. Only like 30, 40 people showed up,
but a dozen people asked me for my slides. But most of my slides only have three or four
words on them, so I decided to record a video of all the things I actually said in my talk.
So this is as best as I can recollect my talk. So, wow, I'm going to skip that. That was
the Twitter hashtag for this talk. Yeah, South by Southwest. So, metafilter is a 12-year-old
community. It's one of the first weblog communities. I was in the right place at the right time.
It's grown by leaps and bounds. The most popular part of the site is the question and answer
area that we call Ask Metafilter. In the last four years, since about 2007, we've doubled
in our traffic, and we've had the same amount of staff since then, and I think it's due
to some of the lessons I'm going to share today. I think any site that's bigger than
a busy WordPress blog all the way up to something probably just below Internet scale, a lot
of these techniques might not work on a massive site like Facebook or YouTube or something,
but for anyone in between, I think there's a lot of lessons here.
So running a community is not easy. You have a lot of people that are fans of the site
and also critics of the site, and as it grows, it becomes harder and harder to control. This
is the entire staff of Metafilter. It's just four of us, three moderators, one programmer,
and we handle up to, I think we're over 22 million pages a month by about 8 million different
people, and that's the viewer level. There's about 12,000 to 50,000 accounts of various
activity on the site that are actually posting stuff.
I like to tell friends that as big as the monster traffic the site gets, the entire
staff couldn't even win a lowly baseball game against these guys.
So at Metafilter, we've had a super focus on customer service and moderation, and we've
also built up a zillion tools I'm about to show you. My hope is by the end of this that
you can follow our lead, build some tools, figure out ways you can help your own community,
and do your best. So like I said at the start, I kind of consider
this kind of an intermediate level community talk and subject matter. I've given lots
of talks in the beginning of community, and I'll post links to things I've said in the
last few years about the basics of community. I'm trying to keep this at 30 minutes or so,
so there's no real way to cover the entire breadth of community basics in just half
hour. So this is sort of a mid-level talk. I hope the intended audience sees it and gets
something out of it, and if you want some more on the beginning, beginner sort of aspects
of community, I'll provide links for that. So no matter how big a community gets, it's
still important from day one to be active in your community and be the best participant,
be the model citizen, especially as it grows, it becomes harder and harder. Your time will
be pushed and pulled in all sorts of ways. You could lose touch with your product, whatever
that is or site, if you're not actively still using it as much as you could. I just think
and it's always great to have someone in charge of community just mixing it up with everybody
still many years later. I'm going to show you a lot of bad things that can happen to
communities, and I think the first response might be to scale back or implement a bunch
of controls as soon as you can. I would say don't just be overprotective for the sake
of it, even if people have told you horror stories. We've come to the conclusion of putting
up barriers where necessary only after we've been permissive for years and years. I like
to think of this as a concert. You don't want your security at the front of the, between
the band and the crowd, pushing the crowd back. It's not really where you want moderators
to be. You want them to be part of it, participants in it. In this slide, hire these people front
of the mosh pit, the part of the action. They can also keep control of the crowd. Don't
hire that guy. It's kind of a separation. Another important thing for moderators, keep
them sane. Keep them happy. Don't let them burn out. I've seen burnout happen and felt
it myself when I felt like I was just seeing the worst parts of the site. I was just dealing
with complaints all day. I'm just going back through why are they complaining? What happened?
A lot of bad stuff happened. You might lose sight of there's actually good things about
your community. I've heard the same stories about policemen who get the worst beat. All
they do is deal with crime all day. All they interact with is horrible, messed up people,
committing crimes, and they become jaded and hate the world. Our moderators, we don't force
time off, but I think me and the other moderators take weekends off at least once a month and
try to get offline for a day or two each month and to participate in all the good parts of
the site just to remind ourselves that it's not just complaints and fights and stuff we
have to deal with that most people don't see. I just want to give you a few more community
ideas before I dive into some of the particulars. It's really important to give users a place
to event and offer feedback. We have a whole sub-site dedicated to users telling us about
the site and giving us feature requests and reporting bugs and talking about educator
policy issues. We monitor Twitter and Facebook mentions and we answer stuff as much as we
can. We're really vigilant about our contact form. It sounds small, but every time someone
hits a contact form, it's five to ten minutes of work and it happens at least 30 up to 50,
60 times a day. Every 20, 30 minutes during the daylight hours for fielding responses
from users and sometimes those are emailed back and forth to go for hours. We spend a
lot of time listening to users and clearing things up. I think it's important to highlight
the best of your site for prospective new users of your community to show them this
is what we have to offer. Also to give your existing user something that they can strive
towards. This is the best of the best of what we do. It's good to have around, like I said
before, for your moderator sanity at least. That's important to involve users as much
as you can in the process. Like I said, we have a whole sub-site dedicated to fielding
requests from users and we pretty much take feature requests there. We also flip out new
ideas that we've thought of as here's a beta URL, try it out, give us feedback. We have
this whole public process for all this stuff before everybody goes live across the sites.
So I want to switch gears and talk specifically about moderation. I've been thinking for the
last couple of years about what makes good moderators moderation, bad moderation, what
they'll look for in good moderators and bad moderators. Moderation really is all about
pattern recognition. If you look at this slide, it's a tile repeated four times. As I'm talking
about this, your brain is trying to find the same gaps four times in that pattern that's
showing up. That's just like when you're reading a community, especially like Metafilter,
that's just a wall of text with a few user names. That's all we're doing all day. Staring
the text and our brains are working and trying to figure out as moderators. The only things
that are shared are user names and we kind of line up user names. Oh, that's that person
I recognize. This is how they're acting or what they're saying. This is what each moderator
of Metafilter has in their head. There's about 125,000 accounts that have ever been created.
About 50,000 were completed and paid and used. At any one time, about 12,000 people have
done something the last few days. About 1,500 different users do something each day, like
add some content to the site. There's roughly 3,400 heavy users. I think all the moderators
can recall all those names if they ran into them at a party or something. These are my
own personal hangups of some of the user names. I always get Slapy, Pinch, Bottom, and Slardy,
Blart, Fast, Mixed Up because one hosts a radio program, the other one hosts about music.
Two active users that have slightly similar names, Pink, Stainless, Tail, and the Pink
Superhero. They go by PST and TPS often on the site. People mix those up and know one
person is a man and one person is a woman. Last week, we got an email from one of those
four Jacqueline saying, I'm not that other Jacqueline. I'm not the most popular or most
active Jacqueline and people are getting this mixed up so often that I'm going to change
my username and get a new account. As moderators, we just have hundreds and hundreds of user
names in our head and thousands of stories for each one of them. It's kind of like a
soap opera about each one of every user because I know one person is divorced in Toronto and
this other person has just lost their job in New York and maybe that's why they might
be acting up. Now we have multiple moderators and we have these hundreds of stories and
personal soap operas in our minds and it becomes really difficult to figure out ways to share
all those weird connections between all the stories and user names on the site and how
they might be behaving. So the core theme of this entire talk that I want you to take
away from it is sit down for a second and figure out the ultimate goal for your community
and that can be like a good thing you want to encourage, highlighting the best material
or elevating the best users or something of that nature or thinking the most annoying
thing about your site. I really hate it when people do X. I really hate it when people
spam the site. I really hate it when people get in fights on the site. I really hate the
way it's hard to read some parts of the site. Whatever it is, especially when it comes to
user behavior, try to notice the patterns in those things that meet your goals. So if
it's like you want to highlight the best posts ever, what's a pattern? Is it number of comments?
Is it the speed of commenting? Is it how often to share it on other networks? Do you have
like favorites or like buttons on it that you can use as a metric? And so it's basically
just coding tools to mimic those patterns. I like to think of it as just building really
dumb AI, artificial intelligence. So just to recap, this is just figure out a goal, figure
out the good and bad things you want on your site elevated and hidden on your site that
are bad and try and figure out what patterns lead up to that and how you can code up a
tool to help highlight those things or push those things down. And I'll show you some
of our tools. These are the tools we've built at Metafilter over the last few years. They've
helped out a lot. When any moderator logs in to the admin backend, this is sort of their
dashboard, and it's been tweaked over the last few years, and I'll just cut through
all the parts. So I have these like one-click links to utilities we use all the time. You
know, this is on the order of several times a day. We have lots of searching, deep searches
on users. I'm constantly looking up if I recognize some weird behaviors coming from
a new user. I'll often look them up and see what their history is on the site. So I'm
using this dozens of times a day. We have some like server health monitors, and we have
this days since we've banned a user or that they've quit the site and closed their account.
And that was just for curiosity's sake. We thought it might be like, I don't know, once
a week or something, you know, like that number would be around 10 most of the times. And it
turned out it's like zero to three all the time. We get a couple spammers a week, and
people shut down their account kind of often for stuff like, well, I have a test coming
up, or my thesis is due, and you know, I need to really force myself off the site and close
my account. So that happens. The main thing on the admin dashboard is this feature of
flagging we have in these hot spots, and I'm going to go deep into this in a little bit.
Our admin tools in general are just anything we could possibly think up will code up, and
we try to keep them all in one place so easy to find, easy to use. So nothing's a hassle
for moderators to get at. This was just a screen of graphing we did after like a New
York Times article came out mentioning Metafilter and Positive Light, and we wanted to see was
there a user-based bump up? Has it been more posts lately or less? Has activity on the
site been spiking because as a result? I think it's important for when you're building
your tools, use whatever preferred mode your moderators like. So I'm a big email guy, still
live sort of in a basic email box, so I get every post email to me the moment it's made.
So I'm dealing with 50, 60, 70 posts a day of all the posts around the site being emailed
to me, except for Ask Metafilter. And it's just I can spot problems immediately, coding
errors that need to be fixed really quick or questionable content. And we also code all
our tools, so there are as many as possible, everything's web-based so we can possibly
make web-based. 5, 10 years ago I was having to open up SQL query analyzers on the desktop
to grab stuff out of the database and fix stuff. So everything's web-based. Everything
is AJAX as much as possible to make sure it all works on like an iPhone. All the moderators
have iPhones and it's sort of a thing. It can be in line at Costco and checking the most
flag things and removing stuff or fixing HTML bugs that people have implemented in the site.
So I would say stick with preferred mode. I could see, you know, if you live and die
by RSS or Twitter or SMS, you can import alerts, sort of go via the preferred method for everyone.
We have this labs area for new tools and we just sort of anything we think of, it gets
built and linked here. And when things are really useful, they sort of get promoted to
the front of the admin dashboard. So for sharing information about users among multiple moderators,
it's sort of built a backend password, built a backend comment system really. So you can
leave a comment about any user and other moderators and come in and see that. So this isn't used
a ton but it's super useful for, you know, this person's been on Rance lately and they
just lost their job. So that's probably what's happening. And then you come back to that user
six months later and they're doing very good things or very bad things. You might look
them up and you'll see their history of what another moderator noticed. And often, these
moderator notes are for alerting other moderators of something that one moderator discovered
like, oh, this person works at NASA and they popped into our shuttle, space shuttle threat
and they gave us some awesome background information. And without reading the entire site, you know,
other moderators might miss those things and so we note those things in their accounts.
All admin activity is logged and searchable in this backend and it's not. So me as the
owner of the site or the main guy in charge, it's not so I can like sort of spy on employees
moderating the site. It's really just so we're all on the same page, you know, you can go
sleep for eight hours or take off for the weekend, come back and check the admin activity
log and see everything that's been deleted or edited and who did it and why. And it's
all searchable. So when someone says, hey, that thing I did two weeks ago, whatever happened
to it, you can do a quick lookup and figure that stuff out. It's super handy.
Our whole point of this is trying to reduce the friction of moderation, streamline the
process as much as possible. Keep all the moderators on the same page and up to date
with the most information possible. This is a screenshot of all the recent user notes.
That's a page you can just look at in our admin backend to just catch up on everything that's
been happening lately and there's good and bad notes there. We use inline tools, wherever
possible. Metafilter is completely custom coded. So we can do anything we want with
our comment system. So this is a screenshot showing a couple grayed out. Those are deleted
comments that are removed so the public isn't seeing them anymore. But we can see them and
we can see who deleted it when. You can see how many flags that item might have gotten.
Every post has an edit and delete control and we also do some IP who is in subnet searching
because we had some identity issues I'll talk about in a second. It's important not to forget
that there's a 24 hour moderator cycle. It's a worldwide web. Your site is always up. I
don't know if enough sites think about this but we have three moderators in America, two
on the west coast, one on the east coast and we have pretty good coverage of the site from
about 6 a.m. to about midnight and we noticed a lot of, we had a string of weird incidents
two or three in the morning that we had to deal with so we ended up getting someone in
London volunteering to just check on the site a couple times in the night for us and the
morning and early afternoon for them and that's pretty much solid all our late night weirdo
problems. If you want to work these in your tools, I don't think, I mean I hope a lot
of these ideas are new to a lot of community owners but it's not impossible. This isn't
rocket science. Metafilter is just written in cold fusion. It was the first web app
I ever wrote 12 years ago. It's really simple stuff for the Microsoft SQL back end. We're
just doing queries, making up these simple SQL queries on our existing database and
really it's just, you've got the database, you just have to think of the pattern, come
up some SQL so if things like the most pop, if you want to list the most popular posts
I know these kinds of things exist for WordPress installations as plugins and stuff but you
can make your own no matter what you're doing if you have access to the database and stuff
like most popular posts might be a number of views of that post plus number of comments
and you can use multipliers and stuff but this is stuff you can just do in one SQL query
really easily and get back results and post that on your site. Just think of the tools,
the hard part is thinking of the tools to highlight good and bad behavior. I think the
actual building of them isn't really that hard. If you ever looked at WordPress databases
there's tons and tons of information there already in an existing structure and just
some simple scripting. I'm sure there's plugins for lots of things and it's not impossible
to write new plugins for whatever it is you want to highlight or hide on your site.
I just want to give you some examples of how we thought through some of these issues. I'll
go into identity. So early on in Metafilter maybe about five years ago there was a period
where we had one problematic user that would constantly appear sort of contrarian to any
discussion that was taking place and then the person had this long history on the site
and they were constantly sort of bickering with other users. A couple of years after
they'd been around someone said, you know, I notice they have these weird spelling inconsistencies
and I'm seeing it show up in some other users and those users happened to defend this one
user. Do you think they might be the same person? I know it's crazy. It turned out one
user was acting like four users on the site. They had three people sort of backing up points
for themselves every so often. They made them seem like they were from all over the world,
male and female and that there was no relationship whatsoever between any of those accounts and
at the time we just did some IP who is lookups and noticed they were all using the same IPs
all the time. And I know you can sort of fake IP address with proxy servers and stuff but
for the most part we sort of have like a dozen sort of identity indicators and who is is
an IP address that you're using is kind of one of those indicators. So we built some
tools or like who has ever used this IP address before, who's used this subnet, who all the
users have ever hit the site on this subnet and sometimes it comes up as useful if we
get some spammy looking thing about some tiny corner of the world and the IP also happens
to be there. The chances that that person is behind the spammy thing is pretty high.
We tracked sock puppets and that's exactly what I described. Someone acting like several
people would be like a sock puppet in community terminology. We are lucky in that we have
this $5 one-time cover charge for Metafilter and it's through PayPal so PayPal gives us
back bunch of information like someone's real name registered PayPal and where they come
from and their business name and their registered PayPal address and you can fake those details.
It's kind of a pain to have multiple PayPal accounts if you really want to hide. But it
works out pretty well for us for finding, we can tell when someone's signing up a second
account or buying one for a friend that's not a sock puppet. We also keep tabs you see
some different colors in this slide and we have another thing which is a watch list.
The red is the person's already been banned. The yellow is this might be a sock puppet
because there's sort of collision with another user and information. The orange stuff is
a watch list and whenever we see we sort of scan new users as they come in. This is actually
a screenshot of our recent signups tool and we'll scan new users coming in and sometimes
they'll just jump out like whatever their email address is. Maybe they're blah blah
blah at goldmoney.biz.ru or something like that where it's just like well I don't know
if they're going to be the best community contributors. I think they might be here to
try and sell something so they go on a watch list and then we have all these tools that
filter out all the users on the watch list. So it's every piece of new content they've
posted and also we're like are they using links in a bad way? Are they trying to link
to sites? You know give us a feed of that. We have feeds like that also in the back end.
So we mostly are using an identity lately to find shills and that's like people we have
a big blanket rule at Metafilter that you're not there to self promote anything. You're
not allowed to self promote whatever company you work for and we just want to keep people
honest and want to make sure whatever they're contributing to Metafilter is because they
really like it and it's not because they're getting paid to do it or they're trying to
make a buck out of it. So a lot of people ignore that rule and so we use all these sort
of and this and like half a dozen other identity tools basically to find people shilling for
their company and so these are just some screenshots of stuff we do. You know we look at whatever
their PayPal records are like you know the ownership of their domain and where their
servers are located and we have to check you know where they're doing online as well. Are
they on Twitter and Facebook? Are they also pimping the same thing? Like you know is this
person a comment spammer or not? It's a lot of the identity work we do. So I mentioned
flagging before so this is a flagging we use for bad stuff. People flagging things as offensive
or that needs attention and we use favorites on the flip side for stuff people like and
you know that's kind of becoming ubiquitous on the web and we've had both these things
flagging since early 2005 and favorites I think we had in 2007. A quick intro to flagging
if you've never heard of it or thought about it is really like if you're been in a drive
through and you see a sign that says something like or at a register and says did you get
a receipt? If you didn't get a receipt tell a manager and we'll give you your meal for
free or we'll give you $5 or something like that. And that's really because the owner
of the store or the restaurant or whatever can't be there 24 hours a day, 7 days a week.
So they're basically asking all their customers to kind of do a little bit of security for
them because if you're not getting a receipt you know a dishonest employee can be saying
like that'll be $5 and doesn't punch it up, takes the $5 puts it in their pocket and gives
the person whatever they ordered and that's called skimming the register. Basically stealing
from the owner. So this is sort of a way to get anyone to report that sort of behavior
on behalf of the owner without the owner having to be around all the time. And the first place
I saw it really was Craigslist and the upper right of any item on Craigslist you can flag
it as offensive or needs attention or formatting bug or this is actually great. It should go
on the best of Craigslist. So you start collecting all these, you know, you let users anonymously
report anything and then you just start running metrics on that stuff. So we just did an implementation
similar to Craigslist. You click a little, we just used an exclamation point at the tail
end of any piece of content on Metafilter that launches a little DHTML pop up and you
can pick out a flag and you can click it and choose it and it's all in AJAX so you never
leave your place and it's a really low friction for the user. And then we just run metrics
on all the flags as they come in. So the first thing we did was just give me a list of all
the flags as they come in and this is sort of like the matrix, you know, they just pop
in every 30 seconds and they're all over the place and you don't know what to make of it
at first. And you just sort of look for repeats of items, you know, they're being flagged
several times and, you know, that's, it's hard to do just by site alone so that's what
the flagging hotspots really is. It's sort of a, we're grouping those similar ones into
where the most flagged items and it's sort of like a velocity of flagging. It's over
a time period so this flag hotspots is really like, you know, give me stuff in the last
few hours that's been flagged the most and just order it by the most flags. And this
is like, these are the things that need your attention, you know, if they're in the dozens
in a short period of time, you know, get on it quick. None of our stuff is automated.
I know Craigslist kind of does, I've heard does some sort of automated stuff like 50
people hate something, it automatically gets deleted and I've heard of, you know, bad stuff
happening where people are colluding to try and push down stuff they don't like, you
know, trying to get rid of things by having 50 faking boats as fast as possible. This
is all still, you know, you're all still done by hand in that respect but we just use these
as places we should pay attention to. And as moderators before that we, like, I used
to be really stressed out. I call this the sequel query that changed my life, the flagging
hotspots because previously I was getting really burned out. The site's 12 years old,
four or five years into it, I was ready to just give up and the faggings, you know, didn't
come onto the site until about the sixth year I'd been around. I would just wake up, read
the site, there's about 200 threads, about 3,000 comments a day, I would try to read
as much as possible and then I have to reread the things that seemed slightly controversial
when I read them earlier and it's just impossible to keep up with, you know, the hundredth comment
might be a fight between two users that was from eight hours ago and you might totally
miss that stuff but you totally get stressed out wondering, you know, what are all the
things happening all over the site and what can I be doing about them but then the fagging
hotspots just brought it all to us. Every user could report anything they saw as problematic
and it just completely changed my life as a moderator and the other moderators. So really
now it's like we can just scan the hotspots about once an hour and do five minutes of
looking at what's the worst things according to users and then you can spend the other
55 minutes of that hour doing something else, living your life or running an errand or reading
the site for fun and it just cut our stress levels down way down. It's just been great.
Having all these flags and having all this data also allowed us to do things like just
show me the most flagged users who posted something that obtained the most flags in
the last month and we have reports like that, this slide shows, and all the moderators that
have all these hundreds of user names in their head, we know who like the four or five thorns
in our sides are or that the community reacts badly to, but this is SQL telling you here's
your biggest jerks on the site, it's kind of cool. If you notice there's different numbers
of flagged items next to each user and when it's low, when it's only one or two things
they've ever been flagged for, it's usually just a huge grammar problem or HTML bug that
people flag to death, but the ones that have like 25 different things that they were flagged
for a dozen of times, those are people that are just causing a problem everywhere. So
how do we use this? Use this when you're dealing with these users and telling them,
please, you might email them, please tone it down. People aren't taking kindly into
how you've been acting on the site and if you know they're on the most flagged list,
you might do things like say, we give people second and third chances, but after that it
might be time for take a week off from the site and come back and try and be a better
contributor to the site or maybe we've given you five chances, we're tired of it. We've
got in the background, we've got the data to prove it, but this really helps out in
running the site. Oh yeah, the slide was here I was just saying that it's kind of amazing
that you just add like a little UI element and then your database can tell you stuff
about your user base and it's something I just never imagined when we implemented it.
On the flip side of flags are favorites and now Facebook throwing like buttons on everything
becomes pretty obvious what that is and it's just people mark stuff as a favorite that
they like, that they agree with or that they want to keep a stream of to refer back to
later and so we let people, you know, mark anything they like as a favorite, any comment
or post and then we have compiled these lists of popularity, we call it, of most favorite
things and we can get a filtered view of each sub-site by favorites and we have RSS feeds
of most favorite posts and most favorite comments and I read the site a ton, I follow these
feeds and I see comments I missed all the time of just like amazing stories that people
left, you know, buried in some 300 comment thing and 50 people noticed it and marked
it and we promote that kind of stuff. Favorites are a big deal, this is real tattoos on someone
and that's our favorites and favorite and you unfavorite, these are the little Ajax
characters on the site for favoriting, those are cool tattoos. The popular lists are cool
for showing off the best parts of the site and we also run like a monthly podcast with
all the moderators where we just sort of talk about for like an hour or so, our favorite
bits from the site and those popularity lists are often mentioned and that's kind of what
points us to the best stuff often. So lastly I want to show you how we get rid of SEO,
I call them SEO assholes, just comment spammers looking to exploit a community to promote
their little website at the expense of whatever, they don't care about the community really,
they just want to get some link from Google and so this is the kind of stuff we've developed
over the years. In the beginning I think comment spam was really easy and obvious, it's like
machine noise, it's just junk, it's just almost anyone could write a filter for that. Comment
spam has kind of evolved into like more, it's more convincing, it's often like semi, sometimes
semi broken English with a couple links thrown in but it's mostly on topic, sometimes it's
very on topic and it's a little bit harder to do and I think this is where comment anti-spam
tools kind of work and sometimes don't work but this is what comment spam I get on my own
blog comes in as, the type of stuff we're getting on Metafilter is really hard, there's
on topic advice but at the end of it they would always say like you know here's two
sentences about your problem and here's an answer and also I found this great service
called blank that does exactly what you need and worked out great and that stuff's really
hard and it turns out when we do some digging that it turns out they own that site and it
becomes really hard, how do you code around that kind of, I call them fake testimonials,
the easy spam like I said machine noise made for bots, they're trying to convince the Google
bot to follow a link with a certain bit of text. A hard spam, you're tricking humans and
you're trying to trick the Google bot at the same time, really hard to suss out what a
regular comment looks like from a fake testimonial and these fake testimonials I mean I lost sleep
over them for weeks about when I first spotted one because I think they're just terrible
and threaten the entire web like anything with reviews, Amazon, Yelp, TripAdvisor, things
are all subject to exploitation of just fake testimonials and conversely they always include
links and stuff, it's an SEO trick on Metafilter so they're just trying to destroy relevant
search engines and it's just killing discussions when any Q&A site is subject to this kind
of exploit, it sucks. You want real honest people asking real honest questions and getting
real honest answers and advice from regular folks, that's what people want out of a Q&A
site. When there's commercial interests involved and they're hiding it and they're deceiving
about the whole thing and saying oh I found this great thing, it just sucks and it kills
everyone's trust in the internet to be a resource. The first time I saw it I deleted
it immediately but then I was like god if I found one how many more are there, how can
we possibly find these, how can we suss these out? So I really just sort of stayed up a
few nights thinking about what was the pattern of usage that led to that, is there anything
we can suss out and really we found these were happening on old posts so these were
new comments coming into a very old post. Posts on Asmofilter are open for one year
but most are answered in like 0-48 hours, most of your answer is over 95%. SEO jerks
doing topic searches are just going to find stuff two months later, six months later,
almost a year later. So it's going to be an old post with a new comment, it's almost
always by a brand new user and they always include HTML link because they're trying to
trick the Google bot and following stuff. So anyone who's ever done database work can
see that these are kind of natural language statements you can use to build a SQL query.
So you just look for give me a list of all old posts that just happened to get a new
comment today and that happened to be by a new user and included an AHRF in it. That's
exactly what we did. We built this thing called the Ask stragglers tool and so we just look
for this kind of activity, it sets the stuff out, in this slide you can see a couple deleted
comments where people are pimping their stuff. We just check this every morning, see what
came in in the last 24 hours and which ones are suspect, follow the links, do some of
the identity lookups and try and figure out if that's really some honest advice or if
it's a fake testimonial. We catch about 2 or 3 people doing this a week on Metafilter
because it has a high page rank, I think it's 7 out of 10 or something. I'm not a big fan
of most SEO stuff so this is a real thing. I showed this slide to a friend, he thought
I made this up and this is fake and this is done in Photoshop but someone actually has
a way to, you know, a 17 minute video for hacking into Metafilter and getting SEO juice
out of it. Just email this guy and he swears he won't send you spam because he hates spam.
One of my readers actually did this and it all comes down to a trick where he does a
Google site search on Ask Metafilter for a keyword you want to spam and then he looks
for the phrase, this thread is closed in new comments as not being present on the page
to show it's one of those less than one year old threads. Our programmer Paul knows the
pattern right away. This is a really strange search and it gets left behind in our Apache
Logs so we do like Apache Log, we check the text each night for this really strange search
phrase and we can see who's been doing it, throw those users on that same IP on the
watch list and do stuff like that to track people at sleeper spammers that might be showing
up soon. So to recap, running the community is tough especially as it grows. It's all
about identifying patterns good and bad on your site and building those tools to highlight
those patterns so you can highlight the good stuff and hide the bad stuff. Moderators plus
tools, really good tools, really good moderators, gives you like ninja-like abilities. I think
it's what helped Metafilter sort of double its traffic load without having to add five
people as staff and if they're awesome moderators and you have these awesome tools you can have
a great community that's easier to manage for everybody involved. And that's it. Thanks
for listening. I'll answer questions on Vimeo down below. Alright, thanks.
