Harnessing the Power of Request Scoped Beans: @RequestScope in a non web-based Request

Spring, the versatile framework, empowers developers with an array of remarkable features for managing the lifecycle of beans. Typically, Spring handles the bean lifecycle when we utilize annotations like @Service or @Component. As the Spring Container starts up, these beans are created, and upon its shutdown, they are gracefully destroyed. However, beyond these familiar methods, there exists a wealth of possibilities to fine-tune the lifecycle of beans.

In this blog post, we'll dive into an aspect of Spring known as "Spring Bean Scopes." Specifically, we'll explore the versatile capabilities of a request scoped bean and discover how it can be utilized beyond the boundaries of a web-based request.

Addressing the Challenge of RequestScope

RequestScope operates seamlessly within a web-based context when handled by the Spring DispatcherServlet. However, it poses an issue when attempting to access a request scoped bean outside the bounds of a web request. You may encounter an exception similar to the following:
‍
java.lang.IllegalStateException: No thread-bound request found: Are you referring to request attributes outside of an actual web request, or processing a request outside of the originally receiving thread? If you are actually operating within a web request and still receive this message, your code is probably running outside of DispatcherServlet/DispatcherPortlet: In this case, use RequestContextListener or RequestContextFilter to expose the current request.

Nevertheless, this limitation is not a flaw in Spring's design. The framework does not expose the RequestScope since it cannot determine when a request starts and ends outside of the DispatcherServlet. Instead, it expects developers to take charge of this responsibility, and we can achieve this through the following steps:

Create a CustomRequestScopeAttr (Mostly copied from blog post of Pranav Maniar)

import org.springframework.web.context.request.RequestAttributes;

import java.util.HashMap;
import java.util.Map;

public class CustomRequestScopeAttr implements RequestAttributes {
    private final Map requestAttributeMap = new HashMap<>();

    @Override
    public Object getAttribute(String name, int scope) {
        if (scope == RequestAttributes.SCOPE_REQUEST) {
            return requestAttributeMap.get(name);
        }

        return null;
    }

    @Override
    public void setAttribute(String name, Object value, int scope) {
        if (scope == RequestAttributes.SCOPE_REQUEST) {
            requestAttributeMap.put(name, value);
        }
    }

    @Override
    public void removeAttribute(String name, int scope) {
        if (scope == RequestAttributes.SCOPE_REQUEST) {
            requestAttributeMap.remove(name);
        }
    }

    @Override
    public String[] getAttributeNames(int scope) {
        if (scope == RequestAttributes.SCOPE_REQUEST) {
            return requestAttributeMap
                    .keySet()
                    .toArray(new String[0]);
        }

        return new String[0];
    }
 // todo implement other methods (not used. just return null)
} 

This will only be available and working for request scoped beans and not for other scopes as well.
‍
Next we need to manually set the RequestAttributes. This needs to be done where logically our request scoped process starts. Its done by calling:

RequestContextHolder.setRequestAttributes(new CustomRequestScopeAttr())  

Once the request scoped process is finished, we reset the RequestAttributes by calling:

RequestContextHolder.resetRequestAttributes();

By calling these methods RequestContextHolder.setRequestAttributes(new CustomRequestScopeAttr()) to start the process and RequestContextHolder.resetRequestAttributes() to end it - we take control of determining when a request begins and concludes.

In the following sections, we will explore various scenarios to illustrate how we can determine the start and end of a request effectively.

Using Request Scoped Beans in @Async

Encountering an @Async annotation within a web-based request can lead to an IllegalStateException when attempting to access a request scoped bean. To overcome this, we need to set the RequestAttributes when creating the async thread. This involves creating a custom AsyncConfiguration to manage async thread creation, where we set the RequestAttributes accordingly.

Following is an example of an AsyncConfiguration:

@EnableAsync
@EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {
    private final TaskExecutionProperties taskExecutionProperties;

    @Override
    @Bean(name = "taskExecutor")
    public Executor getAsyncExecutor() {
        log.debug("Creating Async Task Executor");

        return getExecutor(
                taskExecutionProperties.getPool(),
                taskExecutionProperties.getThreadNamePrefix()
        );
    }
    private Executor getExecutor(TaskExecutionProperties.Pool pool, String threadNamePrefix) {
        ContextAwarePoolExecutor executor = new ContextAwarePoolExecutor();

        executor.setCorePoolSize(pool.getCoreSize());
        executor.setMaxPoolSize(pool.getMaxSize());
        executor.setQueueCapacity(pool.getQueueCapacity());
        executor.setThreadNamePrefix(threadNamePrefix);

        return executor;
    }

When you enter an @Async block, getAsyncExecutor() will be called, and "taskExecutor" is the default name. Alternatively, you can pass an executor name inside the @Async annotation. The configuration is then passed to our custom ContextAwarePoolExecutor, which is defined as:

import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.web.context.request.RequestContextHolder;

import java.util.concurrent.Callable;
import java.util.concurrent.Future;

public class ContextAwarePoolExecutor extends ThreadPoolTaskExecutor {
    /**
     * @param task the {@code Callable} to execute (never {@code null}) - is the actual method we want to call
     */
    @Override
    public  Future submit(Callable task) {
        return super.submit(
                new ContextAwareCallable(RequestContextHolder.currentRequestAttributes(), task)
        );
    }

    @Override
    public  ListenableFuture submitListenable(Callable task) {
        return super.submitListenable(
                new ContextAwareCallable(RequestContextHolder.currentRequestAttributes(), task)
        );
    }
}

The ContextAwarePoolExecutor handles the actual execution of the newly created thread and ensures that the RequestAttributes are properly set:

import org.springframework.web.context.request.RequestAttributes;
import org.springframework.web.context.request.RequestContextHolder;

import java.util.concurrent.Callable;

public class ContextAwareCallable implements Callable {
    private final CustomRequestScopeAttr requestAttributes;
    private Callable task;

    public ContextAwareCallable(RequestAttributes requestAttributes, Callable task) {
        this.task = task;
        this.requestAttributes = cloneRequestAttributes(requestAttributes);
    }

    @Override
    public T call() throws Exception {
        try {
            RequestContextHolder.setRequestAttributes(requestAttributes);
            return task.call();
        } finally {
            RequestContextHolder.resetRequestAttributes();
        }
    }

    /**
     * this is needed, because once the main thread is finished, the object may get garbage collected, even if the async thread is not finished
     *
     * @param requestAttributes
     * @return
     */
    private CustomRequestScopeAttr cloneRequestAttributes(RequestAttributes requestAttributes) {
        CustomRequestScopeAttr clonedRequestAttribute = null;

        try {
            clonedRequestAttribute = new CustomRequestScopeAttr();

            for (String name : requestAttributes.getAttributeNames(RequestAttributes.SCOPE_REQUEST)) {
                clonedRequestAttribute.setAttribute(
                        name,
                        requestAttributes.getAttribute(name, RequestAttributes.SCOPE_REQUEST),
                        RequestAttributes.SCOPE_REQUEST
                );
            }
            return clonedRequestAttribute;
        } catch (Exception e) {
            return new CustomRequestScopeAttr();
        }
    }
}

The ContextAwareCallable class implements the Callable interface and handles the execution of the new thread. In its call() method, we set the RequestAttributes to "mark" the start of the request. As we pass the RequestAttributes from the main thread, we need to clone them before setting. This prevents issues where the main thread might finish before the async thread, leading to attribute garbage collection. In the finally-block of the call() method, we "mark" the end of the request by resetting the RequestAttributes. This approach allows for cascading multiple async calls, because request attributes being passed down and always cloned for each new thread.

By using this mechanism, you can seamlessly utilize @Async annotations while retaining the functionality of request scoped beans.

Using Request Scoped Beans with Pub/Sub

The firewall rule for the connection to the backend only applies to a specific tag, which looks like a node. For example: "gke-staging-456b4340-node "However this is a Network Tag, which is on every Compute Instance of the cluster. Therefor the heaWhen working with Pub/Sub consume events, which are requests from a queue rather than web-based interactions, using request scoped beans requires the setting of RequestAttributes. Fortunately, defining the start and end of a request in a Pub/Sub "context" is straightforward, as we have clear visibility into when a call begins and concludes. In our case, we use Spring Cloud GCP Pub/Sub.

We have a PubSubConsumer class which somehow looks like this:

@Slf4j
public abstract class PubSubConsumer { 
...
    public MessageReceiver receiver() {
        return (PubsubMessage message, AckReplyConsumer consumer) -> {
            try {
                String messageString = parseMessageToString(message, consumer);
                if (messageString == null) {
                    return;
                }

                startConsumeProcess(messageString);

                consumer.ack();
            } catch (NackException e) {
                // we are fine. just nack and try again
                log.info("received nack exception. we will nack this queue entry", e);
                consumer.nack();
            } catch (AckException e) {
                // we are fine. we can ack this one
                log.info("received ack exception. we will ack this queue entry", e);
                consumer.ack();
            } catch (Exception e) {
                // we are not fine
                log.error("error while receiving message from subscription {}", subscription, e);
                consumer.nack();
            } finally {
                RequestContextHolder.resetRequestAttributes();
            }
        };
    }

    protected T parseStringToPayloadType(String messageString) {
        try {
            return objectMapper.readValue(messageString, payloadType);
        } catch (IOException e) {
           ...
        }
    }

    protected String parseMessageToString(PubsubMessage message, AckReplyConsumer consumer) {
        log.info("receive message from subscription {} with payload {}", subscription, message);
        if (message == null || message.getData() == null) {
            ...
        }
        return message.getData().toStringUtf8();
    }

    /**
     * actual consumer process logic.
     *
     * @param messageString String content of a message.
     * @throws Exception
     */
    protected void startConsumeProcess(String messageString) throws Exception {
        RequestContextHolder.setRequestAttributes(new CustomRequestScopeAttr());

        T payload = parseStringToPayloadType(messageString);

        setContextVariables(payload);

        consume(payload);
    }
...
}

When starting the consume process in the startConsumeProcess() method, we set the RequestScope attributes. In the finally-block of the actual Receiver, we reset the RequestAttributes. The entire Pub/Sub event, from the beginning of the consumer until the end, forms a complete request, ensuring that the management of request scoped beans works seamlessly within this context.

Even if you invoke an asynchronous method within the consumer, this approach remains effective. Just make sure to implement the changes from the "Async" chapter to handle asynchronous scenarios correctly. By combining these strategies, you can confidently leverage request scoped beans in Pub/Sub events, facilitating robust and efficient processing of messages within your Spring application.

Dealing with Java ParallelStream

By now I sadly have not found a feasible / generic solution to access a request scoped bean inside of a java ParallelStream. The underlying issue lies in the use of a common fork/join pool by Java streams for parallelization. These threads are neither created nor configured by Spring or the developer, unlike what we did in the "Async" section. While you could manually set the request attributes for each ParallelStream, this is not a practical solution, especially when already dealing with multiple ParallelStreams, leading to potential oversights.

Another approach would be to create a custom ForkJoinPool and use this for parallelstreaming. This may look something like this (copied from here):
‍

final int parallelism = 4;
ForkJoinPool forkJoinPool = null;
try {
    forkJoinPool = new ForkJoinPool(parallelism);
    final List primes = forkJoinPool.submit(() ->
        // Parallel task here, for example
        IntStream.range(1, 1_000_000).parallel()
                .filter(PrimesPrint::isPrime)
                .boxed().collect(Collectors.toList())
    ).get();
    System.out.println(primes);
} catch (InterruptedException | ExecutionException e) {
    throw new RuntimeException(e);
} finally {
    if (forkJoinPool != null) {
        forkJoinPool.shutdown();
    }
}

This approach requires explicitly submitting tasks to the custom forkJoinPool each time you wish to perform parallel operations, making the use of ParallelStreams impractical.

We have made the decision to accept the limitation of not being able to use request-scoped beans inside ParallelStreams. Instead, we adopt alternative strategies to achieve our goals without relying on request-scoped beans in these parallel execution scenarios. While this limitation may present certain constraints, understanding and working around it allow us to use ParallelStreams effectively within our Spring application.

Summary

Request scoped beans in Spring offer significant flexibility, extending their usage beyond traditional web-based requests. However, when using these beans outside of the standard web context, developers must take responsibility for defining the start and end of each process. Spring cannot automatically manage this in custom environments or process flows. Throughout various scenarios, such as asynchronous processing, pub/sub events, and custom thread pools, we've demonstrated effective techniques for handling request scoped beans. The only limitation we encountered thus far is with ParallelStreams, where a generic solution for accessing request-scoped beans is still missing for us. Despite this limitation, understanding the constraints and employing alternative strategies empower developers to leverage the full potential of request-scoped beans within their Spring applications.

Kai Müller
Software Engineer