How do you go about determining if some piece of information is valid?
We all have our biases, but enjoying the feeling of being right is pretty near universal.
As a companion to Paranoia: Pathological or Professional? we should probably talk about a structured approach to validating the inputs you’re using.
Attention Conservation Notice:
It’s a holiday weekend night and one of the thirsty boys next door has a truck with a fresh big block. The revving of the engine interrupted my attempt at an end to end rewatch of Outer Range. This will be a Cranky Post™.
Curation:
First, if you’re not storing what you get in an organized fashion, it’s a real stretch to call it intel at all. If you’re depending on your memory you are prey to your own biases, as well as any deception that is in the works. When professional intelligence managers have reason to suspect there is deception occurring, they will insert a filter layer between the collectors and the analysts. Given my very limited stable of taskable flying monkeys, the policy I’ve settled on for the flat, thinly resourced world in which I exist, is to Stop Ingesting Crap.
Usually I’m the one providing validated, curated content to others, but the summer of 2024 is not a “usually” sort of time. There are news stories, court cases, data leaks, people observing online conflicts, and rather than my constantly pulling there are actually multiple rooms pushing stuff at me. This is nice … but also a bit of a load to bear. I’m not complaining, just noting this curious dynamic.
When you get content, how do you weight it? The Admiralty Code provides a framework to do this.
When a person (as opposed to a persona) who works in a specific field, or who has relevant placement and access, turns up with some information that’s somewhere in the A to C range. Occasionally anonymous persona turn up bearing extraordinary gifts, but as a rule they’re in the D to F range.
Confirmation is a multifaceted sort of thing. Sometimes incoming information is just a hint of where to start, and you can do some hands on collection of your own - that’s grade A. When a pseudonym without any history is involved you’ve got an E or F grade source, especially if there’s any sense of urgency, it’s a 5 - 6 in terms of credibility.
This really should be a cube rather than a flat matrix - because putting new information in context is an important means of weighting potential validity as well. If a particular environment has followed a similar path for years, reports of an incremental change to what is a well known pattern are probably true. If it’s a significant departure, innovation does happen, but another set of eyes or a self inspection of things is warranted.
Analysis:
Once you’ve got a pile of bones, you have to sit down and decide if you’ve got a dinosaur or a Dodo bird. There’s a book on my tradecraft shelf called Structured Analytical Techniques. It’s meant for teams rather than individuals, but I still got a lot of value reviewing it. If you’re cringing at the $50 price for Kindle there’s an earlier book Richards J. Heuer entitled Psychology of Intelligence Analysis which is available gratis and it is a worthy read.
If you’re doing this at all it’s because the breadth and depth of what interests you can not be digested in a glance. Despite all our advances, we remain clever apes meant for hunter gatherer band life on open savanna. Those snap judgments, which have served us well since mother lions were serving up our kind as dinner, are either going to break when we try to scale up, or they’re an attack surface if deception is in the air.
A single folder full of screen shots is a starting point. Getting some labeling going, maybe even some folders, that’s better. Figure out how you’re going to keep things in order time wise.
This is from a chonker of a leak that’s been getting some attention. All the files were date stamped for the day they were liberated, but 60% had easily accessible date stamps as part of their format. I think I know what to do about the 433k I missed in the first round, but I’m dreading it. This Disinfodrome instance has been grinding on that stuff since May 9th.
I’ve been keeping a graph of the MAGA world meltdown since the fall of 2020 and it’s gotten totally out of hand. There are 1,418 URLs in there.
Here are the top 44 sources and their counts of reference for that graph.
Twitter gets top billing for two reasons - first because there are people there who keep context, and second because there is a lot of “connective tissue”. What I mean is that there is typically some fact in a given tweet, and it straddles a couple news articles. When I’m processing an update I look at the tweets for the entities involved and if I’ve got some straight news that covers what the tweet represents, it’ll get retired in favor of a more weighty thing that has editorial oversight.
I really should just take a weekend, go through this thing, and see how many Twitter references can be removed. But what they represent is important and it usually won’t make the Washington Post. I looked at all seven Stewartson tweets and it’s stuff like this. He may go too far with conclusions on things, but I 100% trust his ability to be aware and point out infighting in Flynn’s world.
Dealing With Disinformation:
As a rule you want to Stop Ingesting Crap, but what happens when that crap is part of something that does matter? Here’s a primo example - earlier today Jackie had a list of thirteen accounts. I think she has compelled all of them to block her.
The output from those accounts is toxic nonsense, but whispers in the wind say there actually IS a federal task force aimed at online harassment in general. My knowing that was part of the motivation for writing Stochastic Terrorism As A Service. Illuminating a nexus, as I did in that post, makes information available to law enforcement and reporters that might otherwise pass unnoticed.
As I would only be seeking to understand, as opposed to producing material that could be used in a court case, I’d just watch the metadata. Who Jackie finds to be problematic and what they’ve done, put into a coherent context, might actually be useful.
Oversight:
Way back at the start of the year I spent some time Pondering MITRE’s PANOPTIC, which is a framework for describing privacy violations. The month of May has been chock full of privacy fail that facilitated our Malign Influence Operations Safari. This is just the vibe, here in late Q2 of 2024.
Let’s do a little creative visualization based on what I said earlier. Imagine a class A source that shows up with a #2 grade piece of information. If I were permitted to inspect the trail I’d call it a #1 grade, but they have several good reasons for not sharing precisely how they got it. They want to stick a name on a vile persona and if they’re correct the nature of things is such that this will escalate to reporters paying attention.
There’s another class A source who is very familiar with the persona, as well as the real people around it. They agreed that the #2 piece of information was VERY interesting, but were adamant that the identification was incorrect. They cited a variety of long term observations of theirs, and some things we experienced together in years past.
Conclusion:
So what happened here?
Well positioned responsible adult walks in with something low key amazing, a seeming ringer tying a persona to a name.
A second well positioned responsible adult agrees that the find is important, but that what we have is not identification, it’s proof that Alice talks to Bob, and then Bob let something out via a persona that only a couple people would have been able to access.
I could argue both positions, I think the second is closer to the truth, and I do not want to see a faildox here. So I pushed everything I had on Alice, Bob, and the false front into a folder for review.
Today we ARE certain of a thing that wasn’t even speculation ten days ago. If there’s been one slip, perhaps there are others. A larger set of eyes are now looking, but only three of us know precisely why.
There’s a lot of behavior like this which I dismissively describe as “slap fighting”, but that always involves 1) bullshit personas like Individual None 2) doing free floating bullshit unrelated to anything that matters IRL. This is very different, because there’s a clear path to real world consequences, and it won’t be the first time that has happened for people associated with Alice & Bob.
The entire extent of my Twitter use is an egg that lets me see tweets others share without jumping through hoops. The value of that environment declined precipitously when Space Karen bought it, and when API access ended in April of 2023, that really was The End™ in my book.
That being said …
If you’re having trouble seeing where putting a name to a persona could lead, you should go read how Ricky Vaughn, racist troll, became Douglass Mackey, federal inmate.
It’s fairly difficult to get into trouble for what you say, but what one admits to having done by saying too much is an entirely different matter.
Coda:
It’s 0300 and I couldn’t sleep, so I started chasing the Twitter links, working from least to most. I got proper links for @emptywheel and @RVAwonk references. Much of the rest of the small ones are gone - a couple suspended, several more had deleted tweets. And now the citation URL count is 1,404, down from 1,418. A 1% reduction might not seem like much, but the top quality curation discipline is a not insignificant part of that graph’s value. If someone has a question about an entity on it, I can offer quick, accurate answers with citations.
Those URLs, if automatically turned into PDFs, would be perfect for use with GraphRAG. This is where the world is headed - the crapflood from AI is going to be driven to zero value in many areas by careful curation work being made accessible.