Vibecoding an inference chip

(ankitg.me)

57 points | by ankitvgupta 2 hours ago

7 comments

  • cpldcpu 52 minutes ago
    I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

    It seems implementing transformers is the new "hello world" of AI assisted coding.

  • datadrivenangel 1 hour ago
    Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
    • risyachka 1 hour ago
      Fpga is software-driven, model can verify its output easily.

      Zero surprise it did it

    • datadrivenangel 1 hour ago
      it only cost them $10,000 in Claude API usage. :)
      • hmate9 1 hour ago
        You get that much usage in a monthly $200 subscription.
        • dvh 52 minutes ago
          So who's paying the cost?
          • stronglikedan 12 minutes ago
            Everyone else that doesn't use the entirety of the plan they pay for.
          • nyc_data_geek 8 minutes ago
            Retail and society as ANT shares get pulled into index and wealth funds
          • ralph84 19 minutes ago
            API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.
          • exe34 51 minutes ago
            Corporate customers
        • dofm 1 hour ago
          And a kitten.
  • codekansas 1 hour ago
    This is cool

    Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

  • chalkycrimp 1 hour ago
    Good chance I'm not the only one thinking this: share the code
  • spwa4 12 minutes ago
    I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

    (unfortunately for a decent model it will also be price competitive with a big nvidia card)

  • saejox 52 minutes ago
    i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
  • Archit3ch 19 minutes ago
    Now do silicon. ;)