Track macros from a photo: how it works and what it costs in accuracy

You can now snap a photo of a plate and get calories and macros back in two seconds. The accuracy is good enough that most people who switch from manual logging never go back. Here's how the system actually works under the hood, and where it still falls short.

What the AI sees

A modern food-vision model does three things on every photo: identifies the foods on the plate, estimates the relative portion of each, and looks up nutrition data for each item. The output is a calorie + macro estimate for the whole plate.

Identification is the easy part. Common foods — chicken breast, rice, broccoli, oats — are recognised with high confidence. Mixed dishes (a curry, a stew, a casserole) are harder; the model leans on context cues like colour, texture, plate type, and the foods next to them.

Where the accuracy comes from

Portion estimation is the bottleneck. The model uses reference objects — the plate, a fork, your hand if visible — to anchor scale. A 28 cm dinner plate is a strong reference. A 13 cm side plate, less so. From there it estimates depth (how thick is that pile of rice?) using shadow and texture gradient.

Studies of multi-modal vision models on plated meals consistently land in the ±15–20% range for calorie estimates on familiar foods. That's tighter than most humans guess by hand. It's wider than a kitchen scale. The trade is speed for precision — and for everyone outside competition prep, that trade is correct.

Where it still gets things wrong

• Hidden ingredients — the cream in a sauce, the butter in a pan-seared steak, the sugar dissolved into a salad dressing. The model can't see them.
• Restaurant portions — chefs cook in oil and butter at quantities the photo doesn't reveal. A restaurant burger is often 30% denser than a home one of the same size.
• Mixed liquids — soups, smoothies, blended dishes. Without solids to reference, density estimates wobble.
• Unfamiliar cuisines — regional dishes outside the training distribution drop confidence.

What good photo logging looks like

Take the photo from a 45° angle, not directly overhead. Include the whole plate (not zoomed in). Make sure there's a reference object — usually the plate itself works. Don't crop. If a sauce is drizzled, mention it: "chicken with peanut sauce" beats a photo alone.

If the AI's estimate looks off, correct it once. Saying "more like 200 g rice" teaches the model your specific portion habits over time, and the next photo lands closer.

How KusWise handles this

KusWise runs photo, voice, and text logging side-by-side in Telegram. Photo for plates, voice for restaurants where you don't want to wave a phone around, text for repeats. The bot proposes a log; you accept, edit, or reject. Only what you accept is stored — see our privacy policy.

"Perfect logging at zero photos beats imperfect logging at 100 photos. Pick the method that takes 10 seconds and use it."

Try it: open KusWise in Telegram and send /start. The bot walks you through the first photo. If you'd rather see your daily target first, our calculator gives you calories + macros without an account.