[Spring Boot] DynamoDB

Spring Boot

(Update : 2023-08-24)

Language :

Amazon DynamoDB

Fully managed NoSQL Database service

There is no need to manage servers directly with Serverless Instances available only on Amazon Web Services.

It stores data in a key-value structure and supports JSON.

Because there is no schema, we do not define any additional properties other than the default key when creating the table.

Data types are limited to String, Number, and Binary.

It is a variable plan, and charges are charged according to the amount of use such as reading, writing, and storing data.

It is convenient to back up, deploy, and expand.

Spring Boot와 DynamoDB

It is not officially supported by Spring Data. (Link)

@Transaction is not available.

Git Library

  • Boostchicken: third branch library.It has not been updated since 2020.
  • derjust: Not supported from 2.2
  • Latest library?: According to boostchicken Issue (link), awspring is mentioned, but not sure

Docker localstack

It provides an AWS cloud environment that can be tested on local.

It supports most AWS services. (AWS Service Feature Coverage)

docker-compose.yml (example link)

yml

version: "3"
services:
  aws:
    container_name: exam-aws
    image: localstack/localstack
    platform: linux/amd64
    ports:
      - "4566:4566"            # LocalStack Gateway
      - "4510-4559:4510-4559"  # external services port range
    environment:
      - SERVICE=dynamodb       # Services to use
      - DEBUG=1
      - DOCKER_HOST=unix:///var/run/docker.sock
      # Set up to use aws-cli inside the container. Not specified when using awslocal
      - AWS_ACCESS_KEY_ID=dev     
      - AWS_SECRET_ACCESS_KEY=dev
      - AWS_DEFAULT_REGION=ap-northeast-2
    volumes:
      - "./localstack/data:/var/lib/localstack"
      - "./localstack/init/ready.d:/etc/localstack/init/ready.d"
    restart: unless-stopped

create_dynamodb.sh (./localstack/init/ready.d)

text

#!/bin/bash

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name ad_event \
--attribute-definitions \
 AttributeName=id,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name button_event \
--attribute-definitions \
 AttributeName=id,AttributeType=N \
 AttributeName=username,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
 AttributeName=username,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name click_event \
--attribute-definitions \
 AttributeName=id,AttributeType=N \
 AttributeName=reg_dtm,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
 AttributeName=reg_dtm,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name scroll_event \
--attribute-definitions \
 AttributeName=id,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

Spring Boot 1. GIT Library boostchicken

build.gradle.kts

kotlin

implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.533")
implementation("io.github.boostchicken:spring-data-dynamodb:5.2.5")

Config

kotlin

import com.amazonaws.auth.AWSCredentialsProvider
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain
import com.amazonaws.client.builder.AwsClientBuilder
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig
import org.socialsignin.spring.data.dynamodb.repository.config.EnableDynamoDBRepositories
import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.context.annotation.Primary

@Configuration
@EnableDynamoDBRepositories(basePackages = ["com.example.tracker"])
class DynamoDBConfig(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val region: String
) {

    @Bean
    @Primary
    fun dynamoDBMapper(amazonDynamoDB: AmazonDynamoDB, dynamoDBMapperConfig: DynamoDBMapperConfig): DynamoDBMapper {
        return DynamoDBMapper(amazonDynamoDB, dynamoDBMapperConfig)
    }

    @Bean
    @Primary
    fun awsCredentialsProvider(): AWSCredentialsProvider {
        return DefaultAWSCredentialsProviderChain.getInstance() // Explore credentials in order of environment variables, profile files (~/.aws/credentials)
    }

    @Bean
    @Primary
    fun dynamoDBMapperConfig(): DynamoDBMapperConfig {
        return DynamoDBMapperConfig.DEFAULT
    }

    @Bean
    fun amazonDynamoDB(): AmazonDynamoDB {
        return AmazonDynamoDBClientBuilder
            .standard()
            .withEndpointConfiguration(AwsClientBuilder.EndpointConfiguration(endpoint, region))
            .withCredentials(awsCredentialsProvider())
            .build()
    }
}

Domain

kotlin

import com.amazonaws.services.dynamodbv2.datamodeling.*
import com.example.tracker.common.converter.DynamoDbLocalDateTimeConverter
import java.time.LocalDateTime
import java.util.*

/**
 * required: var, default value
 */
@DynamoDBTable(tableName = "ad_event")
data class AdEvent(
    @DynamoDBHashKey
    @DynamoDBAutoGeneratedKey
    var id: String = UUID.randomUUID().toString(),

    @DynamoDBAttribute(attributeName = "session_key")
    var sessionKey: String = "",

    @DynamoDBAttribute(attributeName = "reg_dtm")
    @DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
    var regDtm: LocalDateTime = LocalDateTime.now()
)

Data Type Transformation Settings

DynamoDB attribute types require converters to convert to dates because only String, Number, and Binary exist.

kotlin

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter
import java.time.LocalDateTime

class DynamoDbLocalDateTimeConverter : DynamoDBTypeConverter<String, LocalDateTime> {
    override fun convert(date: LocalDateTime?): String? {
        return date?.toString()
    }

    override fun unconvert(value: String?): LocalDateTime? {
        return value?.let { LocalDateTime.parse(it) }
    }
}

Repository

kotlin

import com.example.tracker.ad.domain.AdEvent
import org.socialsignin.spring.data.dynamodb.repository.EnableScan
import org.springframework.data.repository.CrudRepository
import java.time.LocalDateTime

@EnableScan
interface AdEventRepository : CrudRepository<AdEvent, String> {

    fun findByRegDtmGreaterThan(regDtm: LocalDateTime): List<AdEvent>

}

Service

kotlin

@Service
class AdEventService(
    private val adEventRepository: AdEventRepository
) {

    fun save(sessionKey: String) {
        adEventRepository.save(
            AdEvent(
                sessionKey = sessionKey
            )
        )
    }

    fun findAll(): List<AdEvent> {
        return adEventRepository.findAll().toList()
    }

    fun findAdEvent(regDtm: LocalDateTime): List<AdEvent> {
        return adEventRepository.findByRegDtmGreaterThan(regDtm)
    }

}

Spring Boot 2. GIT Library, Primary Key

build.gradle.kts (same as above example)

kotlin

implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.533")
implementation("io.github.boostchicken:spring-data-dynamodb:5.2.5")

Config

text

Same as Method 1

Domain

kotlin

/**
 * DynamoDB does not have Schema, so it is safe to set the default value to NULL
 */
@DynamoDBTable(tableName = "click_event")
data class ClickEvent(
    @get:DynamoDBHashKey
    var id: Long? = null,

    @get:DynamoDBRangeKey(attributeName = "reg_dtm")
    @DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
    var regDtm: LocalDateTime? = null,

    @DynamoDBAttribute(attributeName = "json_data")
    var jsonData: String? = null
) {

    // Spring Boot 3 (jakarta) response is not yet available.
    @org.springframework.data.annotation.Id
    private var key: PrimaryKey? = null
        get() {
            return PrimaryKey(id, regDtm)
        }

}

Domain Key

kotlin

data class PrimaryKey(
    @DynamoDBHashKey
    var id: Long? = null,

    @DynamoDBRangeKey(attributeName = "reg_dtm")
    @DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
    var regDtm: LocalDateTime? = null
)

Repository

kotlin

package com.example.tracker.click.repository

import com.example.tracker.click.domain.ClickEvent
import com.example.tracker.common.domain.PrimaryKey
import org.socialsignin.spring.data.dynamodb.repository.EnableScan
import org.springframework.data.repository.CrudRepository

@EnableScan
interface ClickEventRepository : CrudRepository<ClickEvent, PrimaryKey> {

    fun findByKey(key: PrimaryKey): List<ClickEvent>

}

Service

kotlin

@Service
class ClickEventService(
    private val clickEventRepository: ClickEventRepository
) {

    fun save() {
        clickEventRepository.save(
            ClickEvent(
                id = 8L,
                regDtm = LocalDateTime.now(),
                jsonData = Gson().toJson(JsonData("abcd"))
            )
        )
    }

    fun findAll(): List<ClickEvent> {
        return clickEventRepository.findAll().toList()
    }

    fun findByKey(key: PrimaryKey): List<ClickEvent> {
        return clickEventRepository.findByKey(key)
    }

}

Spring Boot 3. AWS SDK Kotlin

build.gradle.kts

kotlin

implementation("aws.sdk.kotlin:dynamodb-jvm:0.21.4-beta")
implementation("com.squareup.okhttp3:okhttp:5.0.0-alpha.11")

Client Provider

kotlin

@Component
class DynamoDbClientProvider(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val regions: String
) {
    fun client() = DynamoDbClient {
        region = regions
        endpointUrl = Url.parse(endpoint)
        credentialsProvider = EnvironmentCredentialsProvider()
    }
}

Domain

kotlin

import java.time.LocalDateTime

data class ClickEvent(
    var id: Long? = null,
    val regDtm: LocalDateTime? = null,
    val jsonData: String? = null
)

Repository

kotlin

import aws.sdk.kotlin.services.dynamodb.model.AttributeValue
import aws.sdk.kotlin.services.dynamodb.model.QueryRequest
import com.example.tracker.click.domain.ClickEvent
import com.example.tracker.common.provider.DynamoDbClientProvider
import kotlinx.coroutines.runBlocking
import org.springframework.stereotype.Repository
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

@Repository
class ClickEventRepository(
    private val clientProvider: DynamoDbClientProvider
) {

    companion object {
        const val TABLE_NAME = "click_event"
    }

    fun findByRegDtmBetween(
        partitionVal: Long,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ClickEvent> {
        val list = mutableListOf<ClickEvent>()

        runBlocking {
            val attr = mutableMapOf<String, AttributeValue>()
            attr[":a"] = AttributeValue.N(partitionVal.toString())
            attr[":t1"] = AttributeValue.S(startDtm.toString())
            attr[":t2"] = AttributeValue.S(endDtm.toString())

            val request = QueryRequest {
                tableName = TABLE_NAME
                keyConditionExpression = "id = :a AND reg_dtm BETWEEN :t1 AND :t2"
                expressionAttributeValues = attr
            }
            clientProvider.client().use { ddb ->
                ddb.query(request).items?.forEach {
                    val id = it.getValue("id").asN()
                    val regDtm = it.getValue("reg_dtm").asS()
                    val jsonData = it.getValue("json_data").asS()
                    list.add(
                        ClickEvent(
                            id = id.toLong(),
                            regDtm = LocalDateTime.parse(regDtm, DateTimeFormatter.ISO_DATE_TIME),
                            jsonData = jsonData
                        )
                    )
                }
            }
        }

        return list
    }

}

Service

kotlin

@Service
class ClickEventService(
    private val clickEventRepository: ClickEventRepository
) {
    fun findByRegDtmBetween(
        partitionVal: Long,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ClickEvent> {
        return clickEventRepository.findByRegDtmBetween(partitionVal, startDtm, endDtm)
    }

}

Spring Boot 4. AWS SDK Kotlin 2

build.gradle.kts

kotlin

implementation("aws.sdk.kotlin:dynamodb-jvm:0.21.4-beta")
implementation("com.squareup.okhttp3:okhttp:5.0.0-alpha.11")

service

kotlin

import aws.sdk.kotlin.runtime.auth.credentials.EnvironmentCredentialsProvider
import aws.sdk.kotlin.services.dynamodb.DynamoDbClient
import aws.sdk.kotlin.services.dynamodb.model.*
import aws.smithy.kotlin.runtime.net.Url
import kotlinx.coroutines.runBlocking
import org.springframework.beans.factory.annotation.Value
import org.springframework.stereotype.Service

@Service
class ScrollEventService(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val regions: String
) {
    companion object {
        const val TABLE_NAME = "scroll_event"
    }

    fun getAll() {
        runBlocking {
            val request = ScanRequest {
                tableName = "scroll_event"
            }

            getClient().use { ddb ->
                val response = ddb.scan(request)
                response.items?.forEach { item ->
                    item.keys.forEach { key ->
                        println("key name: $key / value: ${item[key]}")
                    }
                }
            }
        }
    }

    fun putItem(
        key: String,
        keyVal: String,
        columnKey1: String,
        columnKey1Val: String
    ) {
        runBlocking {
            val itemValues = mutableMapOf<String, AttributeValue>()
            itemValues[key] = AttributeValue.S(keyVal)
            itemValues[columnKey1] = AttributeValue.S(columnKey1Val)

            val request = PutItemRequest {
                tableName = TABLE_NAME
                item = itemValues
            }

            getClient().use { ddb ->
                ddb.putItem(request)
                println("put item: $TABLE_NAME / key val: $keyVal")
            }
        }
    }

    /**
     * @param keyName = Required key
     */
    fun getItem(keyName: String, keyVal: String) {
        runBlocking {
            val keyToGet = mutableMapOf<String, AttributeValue>()
            keyToGet[keyName] = AttributeValue.S(keyVal)

            val request = GetItemRequest {
                key = keyToGet
                tableName = TABLE_NAME
            }

            getClient().use { ddb ->
                val returnedItem = ddb.getItem(request)
                val numberMap = returnedItem.item
                numberMap?.forEach { key ->
                    println("key name: ${key.key} / value: ${key.value}")
                }
            }
        }
    }

    fun keyConditionExpressionsForQuery(
        partitionAlias: String,
        partitionKeyName: String,
        partitionKeyVal: String
    ) {
        runBlocking {
            val attrValues = mutableMapOf<String, AttributeValue>()
            attrValues[":$partitionAlias"] = AttributeValue.S(partitionKeyVal)

            val request = QueryRequest {
                tableName = TABLE_NAME
                keyConditionExpression = "$partitionKeyName = :$partitionAlias"
                expressionAttributeValues = attrValues
            }

            getClient().use { ddb ->
                val response = ddb.query(request)
                println("count: ${response.count}")
                response.items?.forEach { item ->
                    item.forEach { map ->
                        println("key: ${map.key} / value: ${map.value}")
                    }
                    println("--------------------")
                }
            }
        }
    }

    fun filterForQuery(
        filterKeyAlias: String,
        filterKey: String,
        filterKeyValAlias: String,
        filterKeyVal: String,
        partitionAlias: String,
        partitionKeyName: String,
        partitionKeyVal: String,
    ) {
        runBlocking {
            val attrNameAlias = mutableMapOf<String, String>()
            attrNameAlias["#${filterKeyAlias}"] = filterKey

            val attrValues = mutableMapOf<String, AttributeValue>()
            attrValues[":$partitionAlias"] = AttributeValue.S(partitionKeyVal)
            attrValues[":$filterKeyValAlias"] = AttributeValue.S(filterKeyVal)

            val request = QueryRequest {
                tableName = TABLE_NAME
                keyConditionExpression = "$partitionKeyName = :$partitionAlias"
                filterExpression = "#${filterKeyAlias} = :$filterKeyValAlias"
                expressionAttributeNames = attrNameAlias
                expressionAttributeValues = attrValues
            }

            getClient().use { ddb ->
                val response = ddb.query(request)
                println("count: ${response.count}")
                response.items?.forEach { item ->
                    item.forEach { map ->
                        println("key: ${map.key} / value: ${map.value}")
                    }
                    println("--------------------")
                }
            }
        }
    }

    private fun getClient() = DynamoDbClient {
        region = regions
        credentialsProvider = EnvironmentCredentialsProvider()
        endpointUrl = Url.parse(endpoint)
    }

}

Spring Boot 5. AWS SDK JAVA

build.gradle.kts

kotlin

implementation("software.amazon.awssdk:dynamodb-enhanced")
implementation("io.reactivex.rxjava2:rxjava:2.2.21")

Config

kotlin

import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedAsyncClient
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient
import java.net.URI

@Configuration
class DynamoDBConfig(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val region: String,
    @Value("\${cloud.aws.access-key-id}") private val accessKeyId: String,
    @Value("\${cloud.aws.secret-access-key}") private val secretAccessKeyId: String,
) {

    @Bean
    fun dynamoDbAsyncClient(): DynamoDbAsyncClient {
        return DynamoDbAsyncClient.builder()
            .region(Region.of(region))
            .endpointOverride(URI.create(endpoint))
            .credentialsProvider(
                StaticCredentialsProvider.create(
                    AwsBasicCredentials.create(
                        accessKeyId,
                        secretAccessKeyId
                    )
                )
            )
            .build()
    }

    @Bean
    fun dynamoDbEnhancedAsyncClient(): DynamoDbEnhancedAsyncClient {
        return DynamoDbEnhancedAsyncClient.builder()
            .dynamoDbClient(dynamoDbAsyncClient())
            .build()
    }

}

Domain

kotlin

import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.LocalDateTimeAttributeConverter
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.*
import java.time.LocalDateTime

@DynamoDbBean
data class ButtonEvent(
    @get:DynamoDbPartitionKey
    var id: Long? = null,

    @get:DynamoDbSortKey
    var username: String? = null,

    @get:DynamoDbAttribute("button_id")
    var buttonId: Long? = null,

    @get:DynamoDbAttribute("reg_dtm")
    @get:DynamoDbConvertedBy(LocalDateTimeAttributeConverter::class)
    var regDtm: LocalDateTime? = null
)

Repository

kotlin

import com.example.tracker.btn.domain.ButtonEvent
import io.reactivex.Flowable
import org.springframework.stereotype.Repository
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedAsyncClient
import software.amazon.awssdk.enhanced.dynamodb.Expression
import software.amazon.awssdk.enhanced.dynamodb.Key
import software.amazon.awssdk.enhanced.dynamodb.TableSchema
import software.amazon.awssdk.enhanced.dynamodb.model.QueryConditional
import software.amazon.awssdk.services.dynamodb.model.AttributeValue
import java.time.LocalDateTime
import java.util.concurrent.CompletableFuture


@Repository
class ButtonEventRepository(
    private val client: DynamoDbEnhancedAsyncClient
) {

    companion object {
        private const val TABLE_NAME = "button_event"
        private val tableSchema = TableSchema.fromBean(ButtonEvent::class.java)
    }

    private val table = client.table(TABLE_NAME, tableSchema)

    fun save(buttonEvent: ButtonEvent) {
        table.putItem(buttonEvent).get()
    }

    fun findByKey(partitionVal: Long, sortVal: String): ButtonEvent? {
        val key = Key.builder().partitionValue(partitionVal).sortValue(sortVal).build()
        val result: CompletableFuture<ButtonEvent>? = table.getItem { it.key(key) }
        return result?.get()
    }

    fun findByKeyAndRegDtm(
        partitionVal: Long,
        sortVal: String,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ButtonEvent> {
        val values = mutableMapOf<String, AttributeValue>()
        values[":start_dtm"] = AttributeValue.builder().s(startDtm.toString()).build()
        values[":end_dtm"] = AttributeValue.builder().s(endDtm.toString()).build()

        val expression = Expression.builder()
            .expression("reg_dtm >= :start_dtm AND reg_dtm <= :end_dtm")
            .expressionValues(values)
            .build()

        val queryConditional = QueryConditional
            .keyEqualTo(Key.builder().partitionValue(partitionVal).sortValue(sortVal).build())

        val result = table.query { r ->
            r.queryConditional(queryConditional)
                .filterExpression(expression)
        }.items()

        return Flowable.fromPublisher(result).toList().blockingGet()
    }

}

Service

kotlin

@Service
class ButtonEventService(
    private val buttonEventRepository: ButtonEventRepository
) {

    fun save(buttonEvent: ButtonEvent) {
        buttonEventRepository.save(buttonEvent)
    }

    fun findByKey(partitionVal: Long, sortVal: String): ButtonEvent? {
        return buttonEventRepository.findByKey(partitionVal, sortVal)
    }

    fun findByKeyAndRegDtm(
        partitionVal: Long,
        sortVal: String,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ButtonEvent> {
        return buttonEventRepository.findByKeyAndRegDtm(partitionVal, sortVal, startDtm, endDtm)
    }

}

Reference

AWS Official Example

AWS Example Kotlin

AWS Example JAVA

AWS Doc SDK Example

https://howtodoinjava.com/spring-boot/access-dynamodb-with-spring

https://betterprogramming.pub/aws-java-sdk-v2-dynamodb-enhanced-client-with-kotlin-spring-boot-application-f880c74193a2

https://kouzie.github.io/aws/aws-dynamodb/#%EA%B8%B0%EB%B3%B8-%ED%82%A4

https://jsonobject.tistory.com/599

https://wedul.site/708

https://dev.to/aws-builders/using-dynamodb-on-spring-boot-featkotlin-3674

https://refactorfirst.com/using-dynamodb-with-spring-boot

https://techfox.tistory.com/45

https://velog.io/@godol811/Spring Boot에서-DynamoDB-aws-java-sdk-v2-Test-code를-이용해서-적용하기

Full example Git link

민갤

Back-End Developer

백엔드 개발자입니다.

ProfileArticles