feat: [OpenAI] PoC - Responses API support with OpenAI SDK Adapter by rpanackal · Pull Request #794 · SAP/ai-sdk-java

rpanackal · 2026-03-25T09:40:00Z

Context

This PoC provides an adapter for Official OpenAI SDK integrations with our SDK by implementing com.openai.core.http.HttpClient. You can find out more about the OpenAI recommended approach here in their docs.

Feature scope:

Introduces a new generic AiCoreOpenAiClient that can work as an adapter for any OpenAI endpoints
- Easy adoption of any openai endpoints (eg: /realtime) that are/will be supported by AiCore
Introduce AiCoreHttpClientImpl
- Implements a new request and response handling pathway (incl. streaming) that is compatible with OpenAI SDK.
- Returns a client with familiar OpenAI SDK API usage for the user.
Constrains allowed endpoint to /response (for now)

Usage

import com.openai.client.OpenAIClient
import com.sap.ai.sdk.foundationmodels.openai.v1.AiCoreOpenAiClient

OpenAiClient client = AiCoreOpenAiClient.forModel(GPT_5)
var request = ResponseCreateParams.builder().input(input).model(ChatModel.GPT_5).build()
Response response = client.responses().create(request)

Pros

Usage via official OpenAI SDK's API; minimal learning curve
Minimal maintenance overhead as we dont need to maintain models as well as client api.
Customisability and ease of integration: By relaxing allowed method, endpoint etc, we can easy support additional openai endpoints (eg /chat/completion, /realtime etc)

Cons

More complex implementation that introduces additional request/response handling pathway in addition to the existing ones
Two different classes for AI Model list which can be confusing
- com.sap.ai.sdk.foundationmodels.openai.OpenAiModel used to select available models in AICore and com.openai.models.ChatModel.ChatModel for request payload configuration in OpenAI SDK
- We can't pick one without loss of semantics

Definition of Done

Functionality scope stated & covered
Tests cover the scope above
Error handling created / updated & covered by the tests above
~~Aligned changes with the JavaScript SDK~~
Documentation updated
Release notes updated

CharlesDuboisSAP

We should:

enable chat completions
enable streaming
have a controller and an updated index.html
deprecate the old client
have a migration guide or link OpenAI's migration guide.

newtork · 2026-03-26T13:54:03Z

    </dependency>
+    <dependency>
+      <groupId>com.openai</groupId>
+      <artifactId>openai-java-core</artifactId>


(Major)

I would argue either new module, or set this dependency as optional

Why? we are going to deprecate the 2024 generated API

I have marked it optional

rpanackal

I want to highlight current api limitation

import static com.sap.ai.sdk.foundationmodels.openai.OpenAiModel;
import com.openai.models.ChatModel;

// Get the client for a deployment by model name and version
OpenAiModel ourAiModel = OpenAiModel.GPT_5
OpenAiClient client = AiCoreOpenAiClient.forModel(ourAiModel) // 

// Supply model again for request payload. Throws without model.
ChatModel openAiModel = ChatModel.GPT_5
var request = ResponseCreateParams.builder().input(input).model(openAiModel).build()

Two sources of truth.

In the current api behaviour, the model in selected deployment takes precedence over the one in payload. But, this behaviour is not apparent to the user.

CharlesDuboisSAP · 2026-03-27T10:16:30Z

+   * @throws DeploymentResolutionException If no running deployment is found for the model.
+   */
+  @Nonnull
+  public static OpenAIClient forModel(@Nonnull final AiModel model) {


You could modify the client to not be instantiated but instead be created at the request level and cached.

No strong preference but it is better API

yes. This is one of the ways along with few other, each with important caveats

Sniffing request: As Charles mentioned we can parse the body in HttpRequest back to JsonNode or request type to infer the model to fetch deployment for - at request time.

As you can imagine, means this deserializing already serialized response.

You will only find model in create() calls. But not in retrieve(), delete() or any other operation. Then how should we fetch a deployment ? just any deployment under foundation-model scenario?

At request time, we can't reliably infer version out of values like "gpt-5-nano", "gpt-5.2", "o3-2025-04-16". AiCore expects distinct fields for model name and model version to match with a deployment. We will have to rework our deployment resolution logic.

Wrapper API: We draft our own wrapper instead of directly returning an object of com.openai.client.OpenAIClient

// Our wrapper client AiCoreBoundOpenAiClient client = AiCoreOpenAiClient.forModel(OpenAiModel.GPT_41); ResponseCreateParams params = ResponseCreateParams.builder() .input("Hello") // .model(...) is optional. We inject or validate for match .build(); Response response = client.responses().create(params);

public interface AiCoreBoundOpenAiClient { AiCoreResponsesService responses(); AiCoreChatCompletionsService chatCompletions(); OpenAIClient raw(); // escape hatch }

Basically, inject model into params, or validate existing model for match with the one in deployment within in our wrapper api.

Maintenance burden is much higher, but we will be able to active choose UX.

newtork · 2026-03-27T11:40:51Z

+    final ClientOptions clientOptions =
+        ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();


I found a way to propagate the model information to the request body.
But it's super ugly :( and you would need to find a way to pass on model information.

(View code suggestion)

Suggested change

final ClientOptions clientOptions =

ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();

final var m = new SimpleModule() {{

setSerializerModifier(new BeanSerializerModifier() {

@Override

@SuppressWarnings("unchecked")

public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription desc, JsonSerializer<?> serializer) {

if (!ResponseCreateParams.Body.class.isAssignableFrom(desc.getBeanClass()))

return serializer;

final var typed = (JsonSerializer<ResponseCreateParams.Body>) serializer;

return new StdSerializer<>(ResponseCreateParams.Body.class) {

@Override

public void serialize(ResponseCreateParams.Body value, JsonGenerator gen, SerializerProvider provider)

throws IOException {

final var buf = new TokenBuffer(gen.getCodec(), false);

typed.serialize(value, buf, provider);

final ObjectNode node = gen.getCodec().readTree(buf.asParser());

if (!node.has("model")) node.put("model", "gpt-5");

gen.writeTree(node);

}

};

}

});

}};

final ClientOptions clientOptions =

ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused")

.jsonMapper((JsonMapper) jsonMapper().registerModule(m))

.build();

I will try this out and get back to you.

I tried to make it work with mixin, without success.

I am suggesting we need to accept the limitation. We could ofc, get clever with propagating model info to request body for Response API since model is optional in it request builder api. But for Chat Completion, the request builder throws right away if model is not provided. So, we can't really provide any convenience there.

Assuming we will deprecate old openai client we have (as per @CharlesDuboisSAP suggestion), we can keep the current state for consistency. Also, add documentation clarifying which will be the authoritative model.

- Streaming no fully enabled

CharlesDuboisSAP · 2026-04-15T08:21:16Z

+   * @return A configured OpenAI client instance.
+   */
+  @Nonnull
+  @SuppressWarnings("PMD.CloseResource")


There is no test that makes sure we don't leave stuff open and we don't have a memory leak

CharlesDuboisSAP · 2026-04-15T08:22:23Z

+    val params =
+        ResponseCreateParams.builder().input(input).model(ChatModel.GPT_5).store(false).build();
+    return CLIENT.responses().create(params);
+  }
+
+  /**
+   * Create a response and immediately retrieve it using the Responses API. This demonstrates the
+   * two-step process of creating and then fetching a response.
+   *
+   * @param input the input text to send to the model
+   * @return the retrieved response object from the Responses API
+   */
+  @Nonnull
+  public Response retrieveResponse(@Nonnull final String input) {
+    // Create a non-persistent response with store=false


Suggested change

val params =

ResponseCreateParams.builder().input(input).model(ChatModel.GPT_5).store(false).build();

return CLIENT.responses().create(params);

}

/**

* Create a response and immediately retrieve it using the Responses API. This demonstrates the

* two-step process of creating and then fetching a response.

*

* @param input the input text to send to the model

* @return the retrieved response object from the Responses API

*/

@Nonnull

public Response retrieveResponse(@Nonnull final String input) {

// Create a non-persistent response with store=false

// Create a non-persistent response with store=false

val params =

ResponseCreateParams.builder().input(input).model(ChatModel.GPT_5).store(false).build();

return CLIENT.responses().create(params);

}

/**

* Create a response and immediately retrieve it using the Responses API. This demonstrates the

* two-step process of creating and then fetching a response.

*

* @param input the input text to send to the model

* @return the retrieved response object from the Responses API

*/

@Nonnull

public Response retrieveResponse(@Nonnull final String input) {

The comment only makes sense next to the function that it refers

CharlesDuboisSAP · 2026-04-15T08:22:55Z

+    assertThat(response.output().get(1).message().get().content().get(0).asOutputText().text())
+        .contains("Paris");
+  }
+
+  @Test
+  @Disabled("Flaky test")
+  void testGetResponse() {
+    final var response = service.retrieveResponse("What is the capital of France?");
+    assertThat(response).isNotNull();
+    assertThat(response.output()).isNotNull();
+    assertThat(response.output().get(1).message().get().content().get(0).asOutputText().text())
+        .contains("Paris");


Suggested change

assertThat(response.output().get(1).message().get().content().get(0).asOutputText().text())

.contains("Paris");

}

@Test

@Disabled("Flaky test")

void testGetResponse() {

final var response = service.retrieveResponse("What is the capital of France?");

assertThat(response).isNotNull();

assertThat(response.output()).isNotNull();

assertThat(response.output().get(1).message().get().content().get(0).asOutputText().text())

.contains("Paris");

val message = response.output().get(1).message();

assertThat(message.isPresent()).isTrue();

assertThat(message.get().content().get(0).asOutputText().text()).contains("Paris");

}

@Test

@Disabled("Flaky test")

void testGetResponse() {

final var response = service.retrieveResponse("What is the capital of France?");

assertThat(response).isNotNull();

assertThat(response.output()).isNotNull();

val message = response.output().get(1).message();

assertThat(message.isPresent()).isTrue();

assertThat(message.get().content().get(0).asOutputText().text()).contains("Paris");

removes 2 warnings

* Minimal new module setup including spec * Generation partial-success * Remove examples * Successfully filter by path * Attach spec filter command * Initial setup * Successful PoC with OpenAI Models * Version 1 * Stable api * Change class name * Add tests * fix dependency analyse issues * Initial draft untested * Second draft - Streaming no fully enabled * Successful E2E * Streaming initial draft * Streaming E2E with chat completion * isStreaming check simplified * Cleanup PoC and rename module * Reduce Javadoc verbosity * Restrict to `/responses` api * Cleanup comments * Charles review suggestions * Charles review - round 2 suggestions * Add dependency * Mark openai dependency optional and new client `@Beta` * Cleanup and no throw on missing model * pmd * Responses API complete * ChatCompletionCreateParams throws without model. Needs rethink client API creation forModel * Cleanup and close with test documenting limitation * First draft responses only * jacoco limits * Remove ResponseService wrapper since remote API behaviour changed

rpanackal changed the title ~~feat: [OpenAI]/poc OpenAI responses apache~~ feat: [OpenAI] PoC: Responses API support with OpenAI SDK Adapter Mar 25, 2026

rpanackal changed the title ~~feat: [OpenAI] PoC: Responses API support with OpenAI SDK Adapter~~ feat: [OpenAI] PoC - Responses API support with OpenAI SDK Adapter Mar 25, 2026

CharlesDuboisSAP reviewed Mar 25, 2026

View reviewed changes

CharlesDuboisSAP reviewed Mar 26, 2026

View reviewed changes

newtork reviewed Mar 26, 2026

View reviewed changes

Comment thread ...n-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/AiCoreOpenAiClient.java Outdated

newtork reviewed Mar 26, 2026

View reviewed changes

rpanackal commented Mar 27, 2026

View reviewed changes

CharlesDuboisSAP reviewed Mar 27, 2026

View reviewed changes

newtork reviewed Mar 27, 2026

View reviewed changes

rpanackal mentioned this pull request Mar 30, 2026

feat: [OpenAI] PoC - An AiCore wrapper for OpenAi implementation #806

Draft

6 tasks

rpanackal added 20 commits April 2, 2026 11:52

Minimal new module setup including spec

27f810c

Generation partial-success

5325a87

Remove examples

b78995b

Successfully filter by path

a9bd960

Attach spec filter command

6488b26

Initial setup

9b92cfe

Successful PoC with OpenAI Models

2cbaa4e

Version 1

65f4131

Stable api

5039f23

Change class name

fd7ebc2

Rebase with main

2441aad

fix dependency analyse issues

ce8e94a

Initial draft untested

e27fcfa

Second draft

cda636f

- Streaming no fully enabled

Successful E2E

44ca574

Streaming initial draft

db16d95

Streaming E2E with chat completion

bdca14c

isStreaming check simplified

0a9e25e

Cleanup PoC and rename module

a829b3a

Reduce Javadoc verbosity

042755c

rpanackal added 8 commits April 2, 2026 11:54

Restrict to /responses api

a5be638

Cleanup comments

e13d605

Charles review suggestions

1931f0f

Charles review - round 2 suggestions

47a396c

Add dependency

6e174f1

Mark openai dependency optional and new client @Beta

4e684be

Javadoc on sample code for store

941fae0

Update allowed endpoints with method check

127bc45

rpanackal force-pushed the feat/poc-openai-responses-apache branch from ccca300 to 127bc45 Compare April 2, 2026 09:54

rpanackal mentioned this pull request Apr 14, 2026

chore: PoC wrapping official OpenAI SDK #822

Draft

9 tasks

rpanackal self-assigned this Apr 14, 2026

CharlesDuboisSAP reviewed Apr 15, 2026

View reviewed changes

		final ClientOptions clientOptions =
		ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();

-    final ClientOptions clientOptions =
-        ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused").build();
+    final var m = new SimpleModule() {{
+        setSerializerModifier(new BeanSerializerModifier() {
+          @Override
+          @SuppressWarnings("unchecked")
+          public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription desc, JsonSerializer<?> serializer) {
+            if (!ResponseCreateParams.Body.class.isAssignableFrom(desc.getBeanClass()))
+              return serializer;
+            final var typed = (JsonSerializer<ResponseCreateParams.Body>) serializer;
+            return new StdSerializer<>(ResponseCreateParams.Body.class) {
+              @Override
+              public void serialize(ResponseCreateParams.Body value, JsonGenerator gen, SerializerProvider provider)
+                  throws IOException {
+                final var buf = new TokenBuffer(gen.getCodec(), false);
+                typed.serialize(value, buf, provider);
+                final ObjectNode node = gen.getCodec().readTree(buf.asParser());
+                if (!node.has("model")) node.put("model", "gpt-5");
+                gen.writeTree(node);
+              }
+            };
+          }
+        });
+      }};
+    final ClientOptions clientOptions =
+        ClientOptions.builder().baseUrl(baseUrl).httpClient(httpClient).apiKey("unused")
+            .jsonMapper((JsonMapper) jsonMapper().registerModule(m))
+            .build();

Conversation

rpanackal commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Feature scope:

Usage

Pros

Cons

Definition of Done

Uh oh!

CharlesDuboisSAP left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rpanackal left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rpanackal Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rpanackal Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rpanackal commented Mar 25, 2026 •

edited

Loading

CharlesDuboisSAP left a comment •

edited

Loading

rpanackal left a comment •

edited

Loading

rpanackal Mar 27, 2026 •

edited

Loading

rpanackal Apr 7, 2026 •

edited

Loading