302 - Private, Local AI: A Contractor's Guide to Open Source Models

Written By Guest User

You want to put AI to work in your business, but every time you start, the same worry stops you: what happens to your data once it's in there? For a lot of contractors, that single fear is the only thing standing between them and real time savings.

In this episode, Khalil and Martin break down the three real security risks with AI, why paying for a team account changes what happens to your data, and when a private, local model is actually worth it. They get into open-source models, HIPAA-sensitive work, tokens and API costs, and what an AI agent really is.

If you've been holding off because you don't trust AI with your information, this is the conversation that shows you the way around it.


Time Stamps

  • 00:39 - Episode Intro

  • 01:09 - Sunsetting GrowthKits

  • 01:54 - Benali Goes AI

  • 03:55 - HIPAA and Secure AI Needs

  • 09:04 - Security Risks and Local Models

  • 28:33 - Skip Fine-Tuning

  • 28:59 - Local Models and Privacy

  • 30:20 - Tokens and API Costs

  • 34:35 - Open Source Model Trust

  • 38:35 - Agents and Adoption

Memorable Quotes

  • "If it's free, you are the product." — Khalil

    "If you're comfortable putting it into a paid Google Drive, you should be comfortable putting it into Claude." — Khalil

    "A lot of roles will go away, but jobs will not go away." — Khalil

    "We are doing it. It's not theoretical." — Martin

    "Just get started, and then all of a sudden you start thinking more and more, and then you're in the game." — Martin

Key Takeaways

  • Stop using free AI tools for company work. Free versions train on everything you upload, and your employees are almost certainly doing it right now without a policy in place.

  • Move to paid team or organization accounts. On an org account, training on your data is off by default, which is the baseline protection most contractors actually need.

  • Write an AI policy and train your team on it. Treat unmanaged AI use like a phishing risk; one person uploading client data to a free account is all it takes.

  • For most contractors, a paid team account is enough. If you're comfortable putting a file in a paid Google Drive, you're safe putting it in Claude or ChatGPT.

  • Consider a private, local model when you handle HIPAA data, trade secrets, or heavy API costs. You can run an open-source model on a server you control with zero data retention.

  • Skip fine-tuning your own model. Almost no contractor has the thousands of past projects it takes to justify it, so focus on using AI well and building tooling around it instead.

  • Find the time and money to start now. Raising prices is usually the fastest way to fund it, and the efficiency gains pay it back.

Resources


More from Martin

More from Khalil

More from The Cash Flow Contractor

Connect with Us

Subscribe to The Cashflow Contractor podcast for more insights on building a profitable contractor business.

Guest User
Next

301 - TIMWOODS for Contractors: The 8 Types of Waste Hiding in Your Business