Why I Built CivicRecon

I met Eric Christen at a fundraiser in San Diego. My dad invited me, and the room was full of people who knew one another. I shook Eric's hand, told him a little about my Marine Corps background and the custom software I was building through Skane Strategies, and we got talking.

Eric had recently attended an AI conference. He had come away with a pretty direct conclusion: “We've been going about this all wrong.”

He was talking about how the Coalition for Fair Employment in Construction (CFEC) tracked public agencies for project labor agreements, or PLAs. CFEC had a list of about 260 public meeting bodies. Someone had to check their agendas, read through the material, and identify agreements that mattered to the organization's work. Their researcher, Kevin Dayton, had spent more than a decade devoting his weekends to that process. Every weekend. That meant time he couldn't spend traveling or relaxing with his wife.

Eric wanted to know whether software could do the tedious part. I asked for the list and got to work.

First, reconnaissance

I didn't start by writing code. I opened every single listing myself.

I found Legistar, eScribe, ordinary HTML pages, Google Drive folders, and a mess of other publishing systems. Some agencies linked directly to an agenda. Others linked to a PDF that told you to click another link to get the actual agenda. Some PDFs were just scanned images. You couldn't search the text until you ran optical character recognition, or OCR.

Before I built anything, I categorized the platforms, file types, and access conditions. I put human eyes on all 260. Ironically, the first thing I did to build CivicRecon was reconnaissance.

My goal wasn't twenty different applications that I would have to babysit individually. I wanted one master system with the tools to retrieve and read agendas, whether the source was a simple webpage or a JavaScript-heavy publishing platform.

“It works on my machine”

At first, it did work on my machine. That's the classic software developer line, of course.

But I couldn't rely on a personal computer that I regularly restarted and used for other things. I rented a server. Immediately, I discovered that some sites blocked entire ranges of data-center IP addresses. Pages and documents that I could access at home suddenly became inaccessible from the server.

I started building platform-specific adapters inside one unified system. Different sites required different rules for locating meeting dates, agenda links, and supporting files. There were also hard constraints on speed, accuracy, and cost. I knew what CFEC was paying for manual research. I wanted the software to operate at or below that cost, which meant I couldn't throw an expensive language-model API call at every decision.

Then Eric asked me on a Friday whether I could have it ready by Monday.

I don't know what the hell I was thinking, but I confidently said yes.

I barely slept for two nights. I built more than twenty adapters, and each one seemed to reveal another problem. One platform's unusually large agenda packets exposed weaknesses in the main processing logic. That was the beginning of what we called the big_file problem. At first, roughly ten megabytes was enough to put a document in a different processing category. Large documents needed to be divided into manageable pieces without losing the information that made them useful. Language models had input limits, and the server had memory limits. Both mattered.

I was solving problems at the level of individual PDF pages and at the level of the entire server, sometimes in the same hour.

The first Monday

By Monday, CivicRecon wasn't polished. Not even close. But I had an organized list of hits, in the double digits, and I emailed it to Eric before our weekly call with CFEC stakeholders.

Then we compared my findings with Kevin Dayton's manual research.

The lists matched.

One early hit I remember was a PLA associated with the Port of Hueneme, Oxnard Harbor District, for demolition and paving work at the Naval Civil Engineering Lab.

Kevin laughed and said, “Well, it looks like I'm out of a job, guys.”

I knew we had a lot of work left, but after two nights with almost no sleep, that was a hell of a payoff. The software had reproduced the findings of a researcher who knew this work inside and out.

From 260 to nearly 2,000

Eric and CFEC's board started thinking bigger almost immediately. Why stop at 260? California has thousands of special districts alone. We could monitor much more of the state.

I said yes again, not fully appreciating what I was taking on. But I think saying yes before you've solved every problem is part of doing anything worthwhile.

We prioritized school districts and community college districts, where CFEC was especially interested in PLAs, community workforce agreements, and related labor provisions. I used public directories and basic scraping tools to identify meeting listings, then added roughly a hundred entities a week. I was working seven days a week.

Scaling exposed a new class of problems. Some districts organized their meeting pages by school year, so a link that worked for 2025–26 might silently stop being useful when the 2026–27 page appeared. We needed a way to catch that.

Larger packets exhausted the server's memory. Scanning more entities took longer, and results became less timely. I added workers that could claim different jobs, then had to fix race conditions so they didn't interfere with one another. Some packets ran to more than 2,000 pages. Enter the monster_file problem. A single one could take forty-five minutes to read, and several arriving together could choke the system.

Every week something broke. Every week I fixed it. I loved that challenge.

The computational bill grew along with the coverage. Ten times the workload could mean roughly ten times the language-model cost. We moved away from relying on expensive flagship Anthropic models for the bulk of the work and found a different approach that cut our processing costs by about 90 percent at scale. That savings let us read more material, which naturally pushed the bill upward again.

I built the system myself with the help of AI coding tools. But an AI assistant doesn't make the architectural decisions for you. I was the one deciding how to split files, schedule jobs, recover from failures, and keep the system useful without spending a fortune.

We also added human review. Today, two reviewers work through assigned batches so that our roughly 2,000-entity database is manually checked on a recurring cycle. The point is to catch broken links, changed publishing practices, and other things that automation can miss. The published service also uses human review of relevant matches before reporting them.

“Surely not.”

The engineering wasn't the only thing that surprised me. What I found about public transparency was worse.

I'm not going to name names or point fingers at individual agencies. But while auditing meeting listings across California, I found dozens of Brown Act violations. I mean exactly that.

One agency's website said agendas were posted physically around its building, rather than online. My reaction was, “Surely not.”

I went and read the law. California Government Code section 54954.2 requires covered local legislative bodies to post regular-meeting agendas at least 72 hours in advance in a publicly accessible place and on the agency's website, if it has one. The online-posting requirement has defined coverage, but for the agencies it covers, pointing residents to a noticeboard or a front desk is no substitute.

I made dozens of phone calls. I would introduce myself as a constituent, explain that I couldn't find the agenda, and ask where it was. Sometimes I was told to pick up a copy at the office. I'd politely explain the Brown Act requirement and recommend that they start posting the agendas online.

Then I'd check back days later. On multiple occasions, the agendas had started appearing online.

There I was, in my chair at my desk, helping drive civic transparency in California with a phone call.

I also found agendas posted late for regular meetings, sometimes around sixty hours before the meeting instead of the required seventy-two. I saw agendas revised after the initial posting, too. Not every amendment is unlawful, but frequent changes create another problem: a researcher who checked a packet once may never see the item added later. No person can reasonably reread thousands of agendas over and over again. That made the case for recurring automated checks even stronger.

Why the work matters to me

I'm proud of the engineering. What started as a weekend sprint for 260 agencies became a system approaching ten times that scope. We kept solving problems that I couldn't have anticipated at the beginning.

I'm also proud of CFEC's mission. I am not anti-union. I'm pro-competition. I believe merit-shop contractors deserve a fair shot at publicly funded work, and I oppose rules that unnecessarily shut qualified contractors out. I see a political cycle in which union money supports politicians who then support policies favoring union contractors. That's my view of the politics behind these fights, and it is one reason I care about the issue.

There are real people on the other side of these agenda items. Electricians, laborers, and their families. Here in San Diego, Berg electric's work at the airport is an example that comes to mind when I think about the importance of having enough qualified contractors available. I have strong views about how PLAs affect competition and project costs, but the specifics of any one airport contract deserve their own documented treatment.

When a board delays consideration of a PLA, or when a contractor has time to speak up before an agreement is adopted, that's meaningful to CFEC. It can preserve an opportunity for merit-shop businesses to compete. Whether any particular agreement increases costs is something to establish with evidence, not a number I need to throw around to make my point.

I didn't spend those sleepless nights just to build a web scraper. I built something that helps people see what government is doing before decisions are made. Sometimes it means catching a proposal buried in a massive packet. Sometimes it means getting an agency to follow a basic posting requirement. Sometimes it means helping a hardworking family keep a fair opportunity to earn a living.

I'm proud of that. And I'm proud that when the engineering got ugly, we never backed down.