Hmmm.... Wondering if this could be eventually used to emulate a PCIe card using another device, like a RaspberryPi or something more powerful... Thinking the idea of a card you could stick in a machine, anything from a 1x to 16x slot, that emulates a network card (you could run VPN or other stuff on the card and offload it from the host) or storage (running something with enough power to run ZFS and a few disks, and show to the host as a single disk, allowing ZFS on devices that would not support it). but this is probably not something easy...
Hi! Author here! You can technically offload the transactions the real driver on your host does to wherever you want really. PCI is very delay-tolerant and it usually negotiates with the device so I see not much of an issue doing that proven that you can efficiently and performantly manage the throughput throughout the architecture. The thing that kinda makes PCIem special is that you are pretty much free to do whatever you want with the accesses the driver does, you have total freedom. I have made a simple NVME controller (With a 1GB drive I basically malloc'd) which pops up on the local PCI bus (And the regular Linux's nvme block driver attaches to it just fine). You can format it, mount it, create files, folders... it's kinda neat. I also have a simple dumb rasteriser that I made inside QEMU that I wanted to write a driver for, but since it doesn't exist, I used PCIem to help me redirect the driver writes to the QEMU instance hosting the card (Thus was able to run software-rendered DOOM, OpenGL 1.X-based Quake and Half-Life ports).
Indeed, and even then, there's some sw-hw-codesign stuff that kinda helps you do what PCIem does but it's usually really pricey; so I kinda thought it'd be a good thing to have for free.
PCIe prototyping is usually not something super straightforward if you don't want to pay hefty sums IME.
Something like the stm32mp2 series of MCUs can run Linux and act as a PCIe endpoint you can control from a kernel module on the MCU. So you can program an arbitrary PCIe device that way (although it won’t be setting any speed records, and I think the PHY might be limited to PCIe 1x)
interesting... x1 would too slow for large amounts of storage, but as a test, a couple small SSDs could potentially be workable... sounds like im doing some digging...
Indeed, the project has gone through a few iterations already (It was first a monolithic kernel module that required a secondary module to call into the API and whatnot). I've went towards a more userspace-friendly usage mainly so that you can iterate your changes much, much faster. Creating the synthetic PCI device is as easy as opening the userspace shim you program, it'll then appear on your bus. When you want to test new changes, you close the shim normally (Effectively removing it from the bus) and you can do this process as many times as needed.
Tangential question: PCIe is a pretty future-proof technology to learn/invest in,
right? As in,
it is very unlikely to become obsolete in the next 5-10 years (like USB)?
Neither of those is going to be obsolete in 5 years. Might get rebadged and a bunch of extensions, but there's such a huge install base that rapid change is unlikely. Neither Firewire nor Thunderbolt unseated USB.
USB4 is the ~third USB protocol stack though (USB1/2 being basically the same iirc, USB3 being a completely separate protocol that neither logically nor physically interacts with USB1/2 at all), heavily based on Thunderbolt to the point of backwards compatibility.
How could you possibly know exactly what protocol they'd be using for the potential future optical PCIe connection? Your guess is as good as anyone's, no?
Curious what you mean by learning? Learning about TLPs? Learning about FPGA DMA Engines like XDMA? Learning about PCIe switches / retimers? Learning about `lspci`?
The other existing solution to this is FPGA cards: https://www.fpgadeveloper.com/list-of-fpga-dev-boards-for-pc... - note the wide spread in price. You then also have to deal with FPGA tooling. The benefit is much better timing.
PCIe prototyping is usually not something super straightforward if you don't want to pay hefty sums IME.
In fact, "zero~th generation" of thunderbolt used optical link, too. Also both thunderbolt and DisplayPort reuse a lot of common elements from PCI-E