Last Updated: April 2026
Groq offers very fast inference for supported LLMs using its LPU hardware and cloud API, aimed at low-latency assistants, agents, and realtime experiences.
Ultra-low-latency LLM inference API on Groq LPU hardware.
Groq offers very fast inference for supported LLMs using its LPU hardware and cloud API, aimed at low-latency assistants, agents, and realtime experiences. A clear strength highlighted in our listing is Standout tokens-per-second for supported models. A likely tradeoff is Model catalog is narrower than giant hyperscaler marketplaces.
Strengths pulled from our listing review and user-facing positioning.
Tradeoffs and limits worth considering before you commit.
ReplicateReplicate runs open-source and commercial machine learning models behind a simple HTTP API with per-second billing, webhooks, and autoscaling so you can add image, video, audio, and language inference without owning GPUs.
FalFal is a generative media inference platform focused on fast diffusion, video, and audio models with serverless endpoints, queues, and workflows tuned for low-latency production apps.
Together AITogether AI provides open-weight and frontier model inference, dedicated endpoints, fine-tuning, and GPU clusters aimed at teams that want open models with serious throughput.
Replicate runs open-source and commercial machine learning models behind a simple HTTP API with per-second billing, webhooks, and autoscaling so you can add image, video, audio, and language inference without owning GPUs.
Fal is a generative media inference platform focused on fast diffusion, video, and audio models with serverless endpoints, queues, and workflows tuned for low-latency production apps.
Together AI provides open-weight and frontier model inference, dedicated endpoints, fine-tuning, and GPU clusters aimed at teams that want open models with serious throughput.
DeepInfra hosts open-weight models behind simple per-token or per-second pricing with autoscaling, aimed at developers who want cheap inference without running their own GPU fleet.
Fireworks AI is a generative inference platform for fast open and proprietary models with serverless deployments, on-demand GPUs, and fine-tuning aimed at production engineering teams.
Modal is a serverless Python platform for running GPUs and CPUs on demand, popular for embedding pipelines, fine-tunes, and custom inference microservices without managing Kubernetes by hand.
RunPod rents GPUs in the cloud with templates for inference, training, and serverless endpoints, aimed at builders who want price-transparent compute.
Baseten helps teams deploy, scale, and monitor custom and open models behind production APIs with autoscaling, observability, and GPU orchestration.
Banana.dev (often paired with Potassium) offers serverless GPU inference for custom models with simple scaling semantics aimed at ML engineers shipping bespoke endpoints.
Hugging Face connects thousands of models to managed inference endpoints and router APIs so teams can serve transformers, diffusion, and embeddings with provider choice behind one integration surface.
Found a useful AI tool? Save this directory or share it with your network to help others discover the future of AI.