Uploading a file with progress in Kotlin

Do you want to upload a file using the clean Retrofit syntax, but aren't sure how to receive the result as well as the upload progress? We will be using Retrofit to perform the file upload, building an implementation that is able to receive the completion progress at intervals and then complete with the remote API response.

Whilst long-running operations are happening, it is nice for the user to see that activity is occurring, such as a progress view being displayed. For the case of a file upload we can show the real progress, which can be represented by the number of bytes transmitted out of the total file size.

We will use the APIs available to us in Retrofit, OkHttp and Okio to build a class that can be used whenever we want a request to publish its progress to whoever wishes to observe it!

Endpoint

We are developing a messaging application that is able to attach a file to a message thread. It is worth noting that we are using Kotlin Coroutines, however, it can be altered to use regular callbacks or a reactive framework such as RxJava.

Our endpoint is a POST request that contains a multipart body, consisting of the filename, file MIME type, file size and the file itself. We can define it using Retrofit, specifying the required parts.

@Multipart
@POST("file")
suspend fun uploadFile(
    @Part("name") filename: RequestBody,
    @Part("type") mimeType: RequestBody,
    @Part("size") fileSize: RequestBody,
    @Part filePart: MultipartBody.Part,
): FileUploadedRemoteDto

Counting progress

If we just wanted to upload the file without any progress, we would simply convert the file to a request body and send it in the request.

fun createUploadRequestBody(file: File, mimeType: String) =
    file.asRequestBody(mimeType.toMediaType())

Monitoring upload progress can be achieved by using our own CountingRequestBody which wraps around the file RequestBody that would have been used before. The data that is transmitted is the same as before, allowing the raw file RequestBody to be delegated to for the content type and content length.

class CountingRequestBody(
    private val requestBody: RequestBody,
    private val onProgressUpdate: CountingRequestListener,
) : RequestBody() {
    override fun contentType() = requestBody.contentType()

    @Throws(IOException::class)
    override fun contentLength() = requestBody.contentLength()

    ...
}

Transmitting the request body is performed by writing it to a Sink, we will wrap the default sink with our own one that counts the bytes that are transmitted and reports them back via a progress callback.

typealias CountingRequestListener = (
    bytesWritten: Long,
    contentLength: Long,
) -> Unit

class CountingSink(
    sink: Sink,
    private val requestBody: RequestBody,
    private val onProgressUpdate: CountingRequestListener,
) : ForwardingSink(sink) {
    private var bytesWritten = 0L

    override fun write(source: Buffer, byteCount: Long) {
        super.write(source, byteCount)

        bytesWritten += byteCount
        onProgressUpdate(
            bytesWritten,
            requestBody.contentLength(),
        )
    }
}

Within CountingRequestBody we can wrap the default sink into our new CountingSink and write to a buffered version of that, in order to both transmit the file and observe its progress.

class CountingRequestBody(
    private val requestBody: RequestBody,
    private val onProgressUpdate: CountingRequestListener,
) : RequestBody() {
    override fun contentType() = requestBody.contentType()

    @Throws(IOException::class)
    override fun contentLength() = requestBody.contentLength()

    @Throws(IOException::class)
    override fun writeTo(sink: BufferedSink) {
        val countingSink = CountingSink(sink, this, onProgressUpdate)
        val bufferedSink = countingSink.buffer()

        requestBody.writeTo(bufferedSink)

        bufferedSink.flush()
    }
}

The result

Whilst observing the upload progress, there will either be progress or a completed response, the perfect candidate for a sealed class. This will allow CountingRequestResult to be the return type and callers can handle both progress updates and the completed result.

sealed class CountingRequestResult<ResultT> {
    data class Progress<ResultT>(
        val progressFraction: Double
    ) : CountingRequestResult<ResultT>()

    data class Completed<ResultT>(
        val result: ResultT
    ) : CountingRequestResult<ResultT>()
}

Perform the upload

Now that we have a way of uploading a file and receiving the upload progress, we can write our FileUploader. Creating the request body for our upload request involves using a CountingRequestBody that reports progress and completion to a MutableStateFlow (or another reactive type).

private fun createUploadRequestBody(
    file: File,
    mimeType: String,
    progressEmitter: MutableStateFlow<Double>,
): RequestBody {
    val fileRequestBody = file.asRequestBody(mimeType.toMediaType())
    return CountingRequestBody(fileRequestBody) { bytesWritten, contentLength ->
        val progress = 1.0 * bytesWritten / contentLength
        progressEmitter.update { progress }
    }
}

The upload request consists of using the Retrofit function we implemented at the beginning, providing the file details and the created request body that will count progress. The Retrofit definition and the format of the request parts will depend on how each particular API is put together. Here we are using a request that contains various plaintext parts for the file details and then one for the file to be uploaded.

private suspend fun performFileUpload(
    filename: String,
    file: File,
    mimeType: String,
    progressEmitter: MutableStateFlow<Double>,
): FileUploadedRemoteDto {
    val requestBody = createUploadRequestBody(file, mimeType, progressEmitter)
    return remoteApi.uploadFile(
        filename = filename.toRequestBody("text/plain".toMediaType()),
        mimeType = mimeType.toRequestBody("text/plain".toMediaType()),
        fileSize = file
            .length()
            .toString()
            .toRequestBody("text/plain".toMediaType()),
        filePart = MultipartBody.Part.createFormData(
            name = "files[]",
            filename = filename,
            body = requestBody,
        )
    )
}

Our main upload function can put together all of these parts to create a single result flow. We will be able to collect this to get progress updates as well as the final result.

fun uploadFile(
    filename: String,
    file: File,
    mimeType: String
): Flow<FileUploadRemoteResult> {
    val progressEmitter = MutableStateFlow(0.0)
    val progressStream = progressEmitter
        .transformWhile<Double, FileUploadRemoteResult> { progress ->
            emit(CountingRequestResult.Progress(progress))
            progress < 1.0
        }
    val resultStream = flow {
        val uploadResult = performFileUpload(
            filename,
            file,
            mimeType,
            progressEmitter,
        )
        emit(CountingRequestResult.Completed(uploadResult.result))
    }
    return merge(progressStream, resultStream)
}

typealias FileUploadRemoteResult = CountingRequestResult<FileUploadedRemoteDto>

We can now upload a file to our API and update a view as the request progresses, which is nice for noticeably long operations like uploading larger files.

uploader.uploadFile(request.filename, request.file, request.mimeType)
    .distinctUntilChanged()
    .collect { uploadResult ->
        // Update progress in UI
        fileUploadingState.update { inProgressState(uploadResult) }

        // Get uploaded file
        if (uploadResult.status is CountingRequestResult.Completed) {
            uploadedFileState.update { uploadResult.status.result }
        }
    }

Wrap up

Monitoring the progress of a web request may not be immediately obvious when reading through the Retrofit API, however, the powerful APIs of OkHttp and Okio can get the job done nicely. The solution we have developed can be used for any web request, as the counting process can be wrapped around any RequestBody that needs to be sent in a request.

I hope the article was useful. If you have any feedback or questions please feel free to reach out.

Thanks for reading!