Session-aware routing
Keep conversations close to their context. Route to a suitable worker with available capacity, while preserving session affinity where possible.
An inference API built for your applications. Perchy operates the models, infrastructure and serving stack; your application connects through authenticated requests.
Perchy provides managed AI inference through authenticated APIs. Customers do not receive SSH, root, operating-system or bare-metal access. The service does not provide model-training or fine-tuning functionality. Server administration remains with Perchy.
API access, dedicated inference capacity and application integration. Every option runs on Perchy-operated infrastructure.
Send authenticated requests and receive model outputs. Perchy runs the models and inference stack, including server administration and maintenance.
Good performance is the sum of many deliberate decisions.
Keep conversations close to their context. Route to a suitable worker with available capacity, while preserving session affinity where possible.
Reuse prefix and session context where supported. Tune cache policy to your request mix, model and memory budget.
Perchy selects model precision, parallelism and serving configuration around the workload, optimising the path from API request to output.
Perchy configures capacity limits, queue policy and timeouts per model to keep busy workloads within the agreed inference capacity.
Measure time to first token, throughput and tail latency. Test the scenarios your users actually experience.
Perchy monitors model capacity, queue depth and worker health to understand performance before changing the serving configuration.
Connect compatible language workloads through an OpenAI-compatible API using service credentials. Keep your application logic close to what you already know.
Compatibility is validated for your model and application. Image and video processing use asynchronous job workflows.
from openai import OpenAI
client = OpenAI(
base_url=PERCHY_API_BASE_URL,
api_key=PERCHY_API_KEY,
)
response = client.chat.completions.create(
model=PERCHY_MODEL_ID,
messages=[{
"role": "user",
"content": "Let's build something extraordinary."
}],
stream=True,
)Integration example. Use the API endpoint, key and model ID issued by Perchy for your agreed service.
Selected internal engineering results.
Real configurations. Clearly scoped measurements.
Generation stage
Official NF4 Turbo12 configuration. One production acceptance sample, including a verified image retrieval. It is an observed run, not a latency percentile or service guarantee.
Download measurement notesYour workload, usage pattern, data boundaries and goals.
A representative test with agreed quality, latency and cost measures.
An agreed service scope, authenticated API access and an integration plan.
Perchy tunes inference serving as your workload and business evolve.
One conversation.
A whole new trajectory.