A Deeper Look at FENNEC's Brain: 3 Modes of Berean Learning

Building F.E.N.N.E.C.™ was a real adventure, partly because I’m such a perfectionist and partly because I am NOT a computer programmer with an engineering degree. I’m not new to nerdy things, and I know how to do technical things like designing websites, using or changing code, debugging stuff and so on. You tend to learn this stuff over time as a content creator solopreneur, but there’s always some new challenge to put you way out of your comfort zone.

For me, creating F.E.N.N.E.C. was just that project.

In the span of about 6 weeks or so, I spent probably about 300 hours building, tweaking, testing, organizing the beta group, creating promotional materials and planning his release. It was more effort than on any other project I had done in the past, and it was pretty intense. But I also learned a lot about AI, structured data, and the intricacies of crafting a truly edifying and fluid learning experience.

In this regard, the magic of F.E.N.N.E.C. is that he is a diverse and versatile companion. My goal wasn’t to make just a simple chatbot that looks up information, but rather one that can do many things and act as a tutor (see what we did there) and as a librarian. To achieve this required a special platform that has the immense functionality I needed to be able to implement both standard RAG (Retrieval-Augmented Generation, or how LLMs look things up), while also having predictable, deterministic elements.

On top of all that, I wanted him to also do everything but in any of the major languages. Talk about perfectionism!

Well, after lots of work and by the grace of God, it all came together. I think if I would have hired a programmer or team — it would have cost me tens of thousands of dollars to build F.E.N.N.E.C. the way I wanted to. I also wouldn’t have been able to do this without the specialized help of AI in general, as I used Claude for the code generations and debugging, but even that is not a perfect solution. LLMs are just giant pattern recognition machines, and this has both its strengths and its weaknesses, and sometimes I would waste hours going in circles on a problem — only to figure out a different approach and get Claude to think differently. Goes to show you that intelligence is not just about patterns or calculation ability ;)

Building F.E.N.N.E.C.’s 3 modes took quite a lot of work because they are each specialized and have their own quirks. Quiz Mode is 100% deterministic, because he pulls a question from hundreds hosted on Github in JSON format and then serves it to you randomly, then grades your answer according to the standard. That required structuring all of my quiz questions (400 total) perfectly with no errors or answer mismatches or strange characters in the data. Then I had to build out the scorecard feature, which gives you a neat emoji-rich progress report after a quiz turn, so you can see how you do (if you want to see it).

Because Quiz Mode is deterministic, the LLM (large language model) is actually never involved. It’s just a series of functions and steps that flow into one another on the backend. But that requires understanding each possible step, and having failsafes for responses that don’t match — for example, when you’re choosing the mode (normal or Fennec Fox Challenge) but say something else, like “tough mode”. This response would trigger a “no match” and you will get reprompted. Originally I had coded “intent” detectors at each of these steps, only to realize that the platform I was using relied on an AI step to do this, and it often misbehaved. Long story short, each step in Quiz Mode had to be built and tested so that it is completely predictable, and that was a ton of work.

Quiz Mode also has an interesting component that connects it to the Librarian Mode. At the end of every turn in F.E.N.N.E.C., you can go and do something else. That way it is a completely fluid experience that caters to the hyper-focused or the hyper-attention deficit and scatterbrained. But to do that required a lot of thinking and careful coding. For Quiz Mode, because it’s deterministic, this required transitioning the quiz answer just given into a KB (knowledge base) query, which meant programming functions to extract the explanation (which all quiz questions have after you answer), detect your intent via “tell me more” or “explain this” triggers — and then take that to the Librarian LLM and use it as a search, so that you can go deeper on a quiz question at any point in time.

The Librarian itself had its fair share of challenges that made me want to pull whatever was left out of my fading hairline. I had spent dozens of hours meticulously organizing my data in JSON format, which is what is useful for this kind of thing. But after endless trial and error, I realized that the way it was being uploaded by default totally ignored all of my structure and “chunked” everything together into arbitrary chunks of X amount of characters.

That may sound like nerd-speak, and it is, but it’s very significant. When F.E.N.N.E.C. looks something up for you, he matches your word pattern with patterns in the knowledge base (KB). But F.E.N.N.E.C. is a research assistant, and he is supposed to cite everything he finds. Doing this is very difficult, because the data has to have citation information for every “chunk” — but the only way to do that is to organize the chunks to each have that kind of data. Pretty obvious, but if the platform is taking everything I have and chunking it in its own notion of chunk size, what happens is that all of that information gets blended together — rendering some chunks without the identifiers and, as a result, confidence scores for the results get reduced drastically.

I noticed this on questions like, “What does Tudor say about the mark of the beast?” because despite this being a huge topic in my library, I would see hallucinations in the resources cited. F.E.N.N.E.C. gets anxious when he doesn’t have enough information, so because of the aforementioned problem, he would make up episode names and URLs and cite them — something we definitely do not want.

Fixing this problem took hours and hours of dialogue with Voiceflow’s support, and trial and error, only to finally find a way to preserve the chunks with identifiers. It was a real challenge, but Librarian Mode still had other kinks to work out besides this. For example, when he finishes a query with you — it’s standard for him to ask a follow-up question. Using the last question on the MOB, he might say something like, “Do you want me to dive into how Tudor links the mark of the beast to the papacy?” In this situation you might be curious, and say “Sure!” — only to get a response like “Sorry, I didn’t understand what you mean. Try again?” And then everyone is confused.

Well, as simple as this seems, it actually isn’t. Separate steps and code have to be put in place to “extract” the follow-up question he provides and send that as a query, after your “sure” or “yes” intent has been detected. If the intent passes, then he does a deeper dive on his follow-up. If it doesn’t pass (because you ask a totally different question), he moves on.

Citations were another headache. Most chatbots on the internet do NOT serve research purposes or offer citations. They are either button-based (totally deterministic) or general training based, where they can answer some questions using search and reasoning. But what if you want the bot to cite his references? Now that’s a whole different ballgame, and the reality is that even a highly rated and versatile platform like Voiceflow is simply not built for the kind of use case that I wanted — which meant having to get very clever about creating standardized, clean and consistent references. The first part was fixed with the data chunking solution, and the other part was putting a whole series of steps AFTER the query (which you don’t see) to extract the resources in the answer and format them into the answer, using another LLM as a processor/translator.

Without getting too complicated, it was… well… complicated!

Last but not least we have Discussion Mode, and I think it’s super cool. Quiz Mode makes all the quiz questions in the Berean QA (and more, as F.E.N.N.E.C. has dozens more than in the book that were added later) come to life in a dynamic and fun way. But I wanted to do the same for the hundreds of discussion questions people had sent me over the years on a variety of topics.

My goal was to put you in the hot seat and make you practice your discernment and knowledge by answering people’s questions, and then leveraging the power of AI to compute whether your answer has a 50% or better match to mine. If it does, you get acknowledgments only. If it doesn’t, then you get positive critiques to help you improve. This is a fantastic way to test your skills and “real world” tactical knowledge, because it is full of challenge and variety on a wide array of topics in faith and life.

Discussion Mode also has a progress report when you reveal my answer (optional), like the Quiz Mode scorecard. But because you can’t finish Discussion Mode in one sitting, I had to develop a way to have the system remember your place and accrue your progress over time — as opposed to quiz questions that reset if you refresh the chat. This involved tweaking the website widget code and going back and forth with support for like 2 weeks, and pulling even more of my precious hair out because solution after solution just wouldn’t get it to do what I wanted.

But eventually it was resolved, and Discussion Mode does retain your progress now. You also are awarded milestones for different tiers of completion, with a special emoji award for the last tier. Although after you finish all of them it will reset the counter, so you can get questions again, the lifetime counter never resets (unless you go incognito in your browser or clear local storage), which means you can have some fun self progress metrics.

Besides all this, making F.E.N.N.E.C. perform everything I just described but in 27 languages other than English was a real adventure. Translator steps with LLM calls and formatter functions had to be implemented in every step where there was text to display. Even with efficient functions, my project has over 100 blocks in it to put F.E.N.N.E.C.’s brain together. That is a HUGE project, so huge even Voiceflow has a reminder to try to reduce block size because it can get visually cluttered once you hit 100.

But imagine what a great tool it is that all of this work can now be made accessible to practically anyone in the world. My goal with F.E.N.N.E.C. was to make a tool that you could have nearby that makes my content super accessible and learning super fun. Whether you are studying alone and learning about a topic, or studying with friends or family and using voice mode to take turns and do quiz questions on a Friday night — this project has unique value for those who love the truth and I hope it will edify countless more.

So that’s a little more about F.E.N.N.E.C.’s brain. It hurts my brain just to think about it now, but I’m glad it’s done. Hopefully everything will continue to work and as this technology improves it can get even better and better, but even for the time being — F.E.N.N.E.C. is pretty sharp and an excellent resource because he is finely tuned. I hope you make use of him and I hope he makes your studying time fun.