[Spring Boot] DynamoDB

Amazon DynamoDB

완전 관리형 NoSQL Database(비관계형 데이터베이스) Service

Amazon Web Services에서만 사용 가능한 Serverless Instance로 서버를 직접 관리할 필요가 없다.

데이터를 Key-Value 구조로 저장하며 JSON을 지원한다.

스키마가 없기 때문에 테이블을 생성할 때 기본 키 외에 추가 속성에 대해서는 정의하지 않는다.

데이터 유형이 String, Number, Binary로 제한되어 있다.

가변 요금제로 데이터 읽기, 쓰기, 저장 등 사용량에 따라 요금이 부과된다.

백업, 배포, 확장이 편리하다.

Spring Boot와 DynamoDB

Spring Data에서 공식 지원하지 않는다. (링크)

@Transaction을 사용할 수 없다.

Git Library

• boostchicken: 3번째 분기된 라이브러리. 2020년 이후 업데이트되지 않고 있다.
• derjust: 2.2부터 지원되지 않는다.
• 최신 라이브러리?: boostchicken Issue(링크)에 따르면 awspring이 언급되지만 확실하지 않음

Docker localstack

local에서 테스트할 수 있는 AWS 클라우드 환경을 제공한다.

AWS 서비스를 대부분 지원한다. (AWS Service Feature Coverage)

docker-compose.yml (example link)

version: "3"
services:
  aws:
    container_name: exam-aws
    image: localstack/localstack
    platform: linux/amd64
    ports:
      - "4566:4566"            # LocalStack Gateway
      - "4510-4559:4510-4559"  # external services port range
    environment:
      - SERVICE=dynamodb       # 사용할 서비스
      - DEBUG=1
      - DOCKER_HOST=unix:///var/run/docker.sock
      # Container 내부에서 aws-cli를 사용하기 위해 설정. awslocal 사용 시 미지정
      - AWS_ACCESS_KEY_ID=dev     
      - AWS_SECRET_ACCESS_KEY=dev
      - AWS_DEFAULT_REGION=ap-northeast-2
    volumes:
      - "./localstack/data:/var/lib/localstack"
      - "./localstack/init/ready.d:/etc/localstack/init/ready.d"
    restart: unless-stopped

create_dynamodb.sh (./localstack/init/ready.d)

#!/bin/bash

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name ad_event \
--attribute-definitions \
 AttributeName=id,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name button_event \
--attribute-definitions \
 AttributeName=id,AttributeType=N \
 AttributeName=username,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
 AttributeName=username,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name click_event \
--attribute-definitions \
 AttributeName=id,AttributeType=N \
 AttributeName=reg_dtm,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
 AttributeName=reg_dtm,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name scroll_event \
--attribute-definitions \
 AttributeName=id,AttributeType=S \
--key-schema \
 AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)

Spring Boot 방법 1. GIT Library boostchicken

build.gradle.kts

implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.533")
implementation("io.github.boostchicken:spring-data-dynamodb:5.2.5")

Config

import com.amazonaws.auth.AWSCredentialsProvider
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain
import com.amazonaws.client.builder.AwsClientBuilder
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig
import org.socialsignin.spring.data.dynamodb.repository.config.EnableDynamoDBRepositories
import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.context.annotation.Primary

@Configuration
@EnableDynamoDBRepositories(basePackages = ["com.example.tracker"])
class DynamoDBConfig(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val region: String
) {

    @Bean
    @Primary
    fun dynamoDBMapper(amazonDynamoDB: AmazonDynamoDB, dynamoDBMapperConfig: DynamoDBMapperConfig): DynamoDBMapper {
        return DynamoDBMapper(amazonDynamoDB, dynamoDBMapperConfig)
    }

    @Bean
    @Primary
    fun awsCredentialsProvider(): AWSCredentialsProvider {
        return DefaultAWSCredentialsProviderChain.getInstance() // 자격 증명을 환경변수, 프로필 파일(~/.aws/credentials) 순으로 탐색
    }

    @Bean
    @Primary
    fun dynamoDBMapperConfig(): DynamoDBMapperConfig {
        return DynamoDBMapperConfig.DEFAULT
    }

    @Bean
    fun amazonDynamoDB(): AmazonDynamoDB {
        return AmazonDynamoDBClientBuilder
            .standard()
            .withEndpointConfiguration(AwsClientBuilder.EndpointConfiguration(endpoint, region))
            .withCredentials(awsCredentialsProvider())
            .build()
    }
}

Domain

import com.amazonaws.services.dynamodbv2.datamodeling.*
import com.example.tracker.common.converter.DynamoDbLocalDateTimeConverter
import java.time.LocalDateTime
import java.util.*

/**
 * required: var, default value
 */
@DynamoDBTable(tableName = "ad_event")
data class AdEvent(
    @DynamoDBHashKey
    @DynamoDBAutoGeneratedKey
    var id: String = UUID.randomUUID().toString(),

    @DynamoDBAttribute(attributeName = "session_key")
    var sessionKey: String = "",

    @DynamoDBAttribute(attributeName = "reg_dtm")
    @DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
    var regDtm: LocalDateTime = LocalDateTime.now()
)

데이터 타입 변환 설정

DynamoDB 속성 타입은 String, Number, Binary만 존재하기 때문에 날짜로 변환하기 위한 Converter가 필요하다

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter
import java.time.LocalDateTime

class DynamoDbLocalDateTimeConverter : DynamoDBTypeConverter<String, LocalDateTime> {
    override fun convert(date: LocalDateTime?): String? {
        return date?.toString()
    }

    override fun unconvert(value: String?): LocalDateTime? {
        return value?.let { LocalDateTime.parse(it) }
    }
}

Repository

import com.example.tracker.ad.domain.AdEvent
import org.socialsignin.spring.data.dynamodb.repository.EnableScan
import org.springframework.data.repository.CrudRepository
import java.time.LocalDateTime

@EnableScan
interface AdEventRepository : CrudRepository<AdEvent, String> {

    fun findByRegDtmGreaterThan(regDtm: LocalDateTime): List<AdEvent>

}

Service

@Service
class AdEventService(
    private val adEventRepository: AdEventRepository
) {

    fun save(sessionKey: String) {
        adEventRepository.save(
            AdEvent(
                sessionKey = sessionKey
            )
        )
    }

    fun findAll(): List<AdEvent> {
        return adEventRepository.findAll().toList()
    }

    fun findAdEvent(regDtm: LocalDateTime): List<AdEvent> {
        return adEventRepository.findByRegDtmGreaterThan(regDtm)
    }

}

Spring Boot 방법 2. GIT Library, Primary Key

build.gradle.kts (위 예시와 동일)

implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.533")
implementation("io.github.boostchicken:spring-data-dynamodb:5.2.5")

Config

방법 1과 동일

Domain

/**
 * DynamoDB는 Schema가 없기 때문에 기본값을 NULL로 하는게 안전
 */
@DynamoDBTable(tableName = "click_event")
data class ClickEvent(
    @get:DynamoDBHashKey
    var id: Long? = null,

    @get:DynamoDBRangeKey(attributeName = "reg_dtm")
    @DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
    var regDtm: LocalDateTime? = null,

    @DynamoDBAttribute(attributeName = "json_data")
    var jsonData: String? = null
) {

    // Spring Boot 3 (jakarta) 대응이 아직 안되어 있다.
    @org.springframework.data.annotation.Id
    private var key: PrimaryKey? = null
        get() {
            return PrimaryKey(id, regDtm)
        }

}

Domain Key

data class PrimaryKey(
    @DynamoDBHashKey
    var id: Long? = null,

    @DynamoDBRangeKey(attributeName = "reg_dtm")
    @DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
    var regDtm: LocalDateTime? = null
)

Repository

package com.example.tracker.click.repository

import com.example.tracker.click.domain.ClickEvent
import com.example.tracker.common.domain.PrimaryKey
import org.socialsignin.spring.data.dynamodb.repository.EnableScan
import org.springframework.data.repository.CrudRepository

@EnableScan
interface ClickEventRepository : CrudRepository<ClickEvent, PrimaryKey> {

    fun findByKey(key: PrimaryKey): List<ClickEvent>

}

Service

@Service
class ClickEventService(
    private val clickEventRepository: ClickEventRepository
) {

    fun save() {
        clickEventRepository.save(
            ClickEvent(
                id = 8L,
                regDtm = LocalDateTime.now(),
                jsonData = Gson().toJson(JsonData("abcd"))
            )
        )
    }

    fun findAll(): List<ClickEvent> {
        return clickEventRepository.findAll().toList()
    }

    fun findByKey(key: PrimaryKey): List<ClickEvent> {
        return clickEventRepository.findByKey(key)
    }

}

Spring Boot 방법 3. AWS SDK Kotlin

build.gradle.kts

implementation("aws.sdk.kotlin:dynamodb-jvm:0.21.4-beta")
implementation("com.squareup.okhttp3:okhttp:5.0.0-alpha.11")

Client Provider

@Component
class DynamoDbClientProvider(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val regions: String
) {
    fun client() = DynamoDbClient {
        region = regions
        endpointUrl = Url.parse(endpoint)
        credentialsProvider = EnvironmentCredentialsProvider()
    }
}

Domain

import java.time.LocalDateTime

data class ClickEvent(
    var id: Long? = null,
    val regDtm: LocalDateTime? = null,
    val jsonData: String? = null
)

Repository

import aws.sdk.kotlin.services.dynamodb.model.AttributeValue
import aws.sdk.kotlin.services.dynamodb.model.QueryRequest
import com.example.tracker.click.domain.ClickEvent
import com.example.tracker.common.provider.DynamoDbClientProvider
import kotlinx.coroutines.runBlocking
import org.springframework.stereotype.Repository
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

@Repository
class ClickEventRepository(
    private val clientProvider: DynamoDbClientProvider
) {

    companion object {
        const val TABLE_NAME = "click_event"
    }

    fun findByRegDtmBetween(
        partitionVal: Long,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ClickEvent> {
        val list = mutableListOf<ClickEvent>()

        runBlocking {
            val attr = mutableMapOf<String, AttributeValue>()
            attr[":a"] = AttributeValue.N(partitionVal.toString())
            attr[":t1"] = AttributeValue.S(startDtm.toString())
            attr[":t2"] = AttributeValue.S(endDtm.toString())

            val request = QueryRequest {
                tableName = TABLE_NAME
                keyConditionExpression = "id = :a AND reg_dtm BETWEEN :t1 AND :t2"
                expressionAttributeValues = attr
            }
            clientProvider.client().use { ddb ->
                ddb.query(request).items?.forEach {
                    val id = it.getValue("id").asN()
                    val regDtm = it.getValue("reg_dtm").asS()
                    val jsonData = it.getValue("json_data").asS()
                    list.add(
                        ClickEvent(
                            id = id.toLong(),
                            regDtm = LocalDateTime.parse(regDtm, DateTimeFormatter.ISO_DATE_TIME),
                            jsonData = jsonData
                        )
                    )
                }
            }
        }

        return list
    }

}

Service

@Service
class ClickEventService(
    private val clickEventRepository: ClickEventRepository
) {
    fun findByRegDtmBetween(
        partitionVal: Long,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ClickEvent> {
        return clickEventRepository.findByRegDtmBetween(partitionVal, startDtm, endDtm)
    }

}

Spring Boot 방법 4. AWS SDK Kotlin 2

build.gradle.kts

implementation("aws.sdk.kotlin:dynamodb-jvm:0.21.4-beta")
implementation("com.squareup.okhttp3:okhttp:5.0.0-alpha.11")

service

import aws.sdk.kotlin.runtime.auth.credentials.EnvironmentCredentialsProvider
import aws.sdk.kotlin.services.dynamodb.DynamoDbClient
import aws.sdk.kotlin.services.dynamodb.model.*
import aws.smithy.kotlin.runtime.net.Url
import kotlinx.coroutines.runBlocking
import org.springframework.beans.factory.annotation.Value
import org.springframework.stereotype.Service

@Service
class ScrollEventService(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val regions: String
) {
    companion object {
        const val TABLE_NAME = "scroll_event"
    }

    fun getAll() {
        runBlocking {
            val request = ScanRequest {
                tableName = "scroll_event"
            }

            getClient().use { ddb ->
                val response = ddb.scan(request)
                response.items?.forEach { item ->
                    item.keys.forEach { key ->
                        println("key name: $key / value: ${item[key]}")
                    }
                }
            }
        }
    }

    fun putItem(
        key: String,
        keyVal: String,
        columnKey1: String,
        columnKey1Val: String
    ) {
        runBlocking {
            val itemValues = mutableMapOf<String, AttributeValue>()
            itemValues[key] = AttributeValue.S(keyVal)
            itemValues[columnKey1] = AttributeValue.S(columnKey1Val)

            val request = PutItemRequest {
                tableName = TABLE_NAME
                item = itemValues
            }

            getClient().use { ddb ->
                ddb.putItem(request)
                println("put item: $TABLE_NAME / key val: $keyVal")
            }
        }
    }

    /**
     * @param keyName = Required key
     */
    fun getItem(keyName: String, keyVal: String) {
        runBlocking {
            val keyToGet = mutableMapOf<String, AttributeValue>()
            keyToGet[keyName] = AttributeValue.S(keyVal)

            val request = GetItemRequest {
                key = keyToGet
                tableName = TABLE_NAME
            }

            getClient().use { ddb ->
                val returnedItem = ddb.getItem(request)
                val numberMap = returnedItem.item
                numberMap?.forEach { key ->
                    println("key name: ${key.key} / value: ${key.value}")
                }
            }
        }
    }

    fun keyConditionExpressionsForQuery(
        partitionAlias: String,
        partitionKeyName: String,
        partitionKeyVal: String
    ) {
        runBlocking {
            val attrValues = mutableMapOf<String, AttributeValue>()
            attrValues[":$partitionAlias"] = AttributeValue.S(partitionKeyVal)

            val request = QueryRequest {
                tableName = TABLE_NAME
                keyConditionExpression = "$partitionKeyName = :$partitionAlias"
                expressionAttributeValues = attrValues
            }

            getClient().use { ddb ->
                val response = ddb.query(request)
                println("count: ${response.count}")
                response.items?.forEach { item ->
                    item.forEach { map ->
                        println("key: ${map.key} / value: ${map.value}")
                    }
                    println("--------------------")
                }
            }
        }
    }

    fun filterForQuery(
        filterKeyAlias: String,
        filterKey: String,
        filterKeyValAlias: String,
        filterKeyVal: String,
        partitionAlias: String,
        partitionKeyName: String,
        partitionKeyVal: String,
    ) {
        runBlocking {
            val attrNameAlias = mutableMapOf<String, String>()
            attrNameAlias["#${filterKeyAlias}"] = filterKey

            val attrValues = mutableMapOf<String, AttributeValue>()
            attrValues[":$partitionAlias"] = AttributeValue.S(partitionKeyVal)
            attrValues[":$filterKeyValAlias"] = AttributeValue.S(filterKeyVal)

            val request = QueryRequest {
                tableName = TABLE_NAME
                keyConditionExpression = "$partitionKeyName = :$partitionAlias"
                filterExpression = "#${filterKeyAlias} = :$filterKeyValAlias"
                expressionAttributeNames = attrNameAlias
                expressionAttributeValues = attrValues
            }

            getClient().use { ddb ->
                val response = ddb.query(request)
                println("count: ${response.count}")
                response.items?.forEach { item ->
                    item.forEach { map ->
                        println("key: ${map.key} / value: ${map.value}")
                    }
                    println("--------------------")
                }
            }
        }
    }

    private fun getClient() = DynamoDbClient {
        region = regions
        credentialsProvider = EnvironmentCredentialsProvider()
        endpointUrl = Url.parse(endpoint)
    }

}

Spring Boot 방법 5. AWS SDK JAVA

build.gradle.kts

implementation("software.amazon.awssdk:dynamodb-enhanced")
implementation("io.reactivex.rxjava2:rxjava:2.2.21")

Config

import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedAsyncClient
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient
import java.net.URI

@Configuration
class DynamoDBConfig(
    @Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
    @Value("\${cloud.aws.region.static}") private val region: String,
    @Value("\${cloud.aws.access-key-id}") private val accessKeyId: String,
    @Value("\${cloud.aws.secret-access-key}") private val secretAccessKeyId: String,
) {

    @Bean
    fun dynamoDbAsyncClient(): DynamoDbAsyncClient {
        return DynamoDbAsyncClient.builder()
            .region(Region.of(region))
            .endpointOverride(URI.create(endpoint))
            .credentialsProvider(
                StaticCredentialsProvider.create(
                    AwsBasicCredentials.create(
                        accessKeyId,
                        secretAccessKeyId
                    )
                )
            )
            .build()
    }

    @Bean
    fun dynamoDbEnhancedAsyncClient(): DynamoDbEnhancedAsyncClient {
        return DynamoDbEnhancedAsyncClient.builder()
            .dynamoDbClient(dynamoDbAsyncClient())
            .build()
    }

}

Domain

import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.LocalDateTimeAttributeConverter
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.*
import java.time.LocalDateTime

@DynamoDbBean
data class ButtonEvent(
    @get:DynamoDbPartitionKey
    var id: Long? = null,

    @get:DynamoDbSortKey
    var username: String? = null,

    @get:DynamoDbAttribute("button_id")
    var buttonId: Long? = null,

    @get:DynamoDbAttribute("reg_dtm")
    @get:DynamoDbConvertedBy(LocalDateTimeAttributeConverter::class)
    var regDtm: LocalDateTime? = null
)

Repository

import com.example.tracker.btn.domain.ButtonEvent
import io.reactivex.Flowable
import org.springframework.stereotype.Repository
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedAsyncClient
import software.amazon.awssdk.enhanced.dynamodb.Expression
import software.amazon.awssdk.enhanced.dynamodb.Key
import software.amazon.awssdk.enhanced.dynamodb.TableSchema
import software.amazon.awssdk.enhanced.dynamodb.model.QueryConditional
import software.amazon.awssdk.services.dynamodb.model.AttributeValue
import java.time.LocalDateTime
import java.util.concurrent.CompletableFuture


@Repository
class ButtonEventRepository(
    private val client: DynamoDbEnhancedAsyncClient
) {

    companion object {
        private const val TABLE_NAME = "button_event"
        private val tableSchema = TableSchema.fromBean(ButtonEvent::class.java)
    }

    private val table = client.table(TABLE_NAME, tableSchema)

    fun save(buttonEvent: ButtonEvent) {
        table.putItem(buttonEvent).get()
    }

    fun findByKey(partitionVal: Long, sortVal: String): ButtonEvent? {
        val key = Key.builder().partitionValue(partitionVal).sortValue(sortVal).build()
        val result: CompletableFuture<ButtonEvent>? = table.getItem { it.key(key) }
        return result?.get()
    }

    fun findByKeyAndRegDtm(
        partitionVal: Long,
        sortVal: String,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ButtonEvent> {
        val values = mutableMapOf<String, AttributeValue>()
        values[":start_dtm"] = AttributeValue.builder().s(startDtm.toString()).build()
        values[":end_dtm"] = AttributeValue.builder().s(endDtm.toString()).build()

        val expression = Expression.builder()
            .expression("reg_dtm >= :start_dtm AND reg_dtm <= :end_dtm")
            .expressionValues(values)
            .build()

        val queryConditional = QueryConditional
            .keyEqualTo(Key.builder().partitionValue(partitionVal).sortValue(sortVal).build())

        val result = table.query { r ->
            r.queryConditional(queryConditional)
                .filterExpression(expression)
        }.items()

        return Flowable.fromPublisher(result).toList().blockingGet()
    }

}

Service

@Service
class ButtonEventService(
    private val buttonEventRepository: ButtonEventRepository
) {

    fun save(buttonEvent: ButtonEvent) {
        buttonEventRepository.save(buttonEvent)
    }

    fun findByKey(partitionVal: Long, sortVal: String): ButtonEvent? {
        return buttonEventRepository.findByKey(partitionVal, sortVal)
    }

    fun findByKeyAndRegDtm(
        partitionVal: Long,
        sortVal: String,
        startDtm: LocalDateTime,
        endDtm: LocalDateTime
    ): List<ButtonEvent> {
        return buttonEventRepository.findByKeyAndRegDtm(partitionVal, sortVal, startDtm, endDtm)
    }

}

Reference

AWS 공식 예제

AWS Example Kotlin

AWS Example JAVA

AWS Doc SDK Example

https://howtodoinjava.com/spring-boot/access-dynamodb-with-spring

https://betterprogramming.pub/aws-java-sdk-v2-dynamodb-enhanced-client-with-kotlin-spring-boot-application-f880c74193a2

https://kouzie.github.io/aws/aws-dynamodb/#%EA%B8%B0%EB%B3%B8-%ED%82%A4

https://jsonobject.tistory.com/599

https://wedul.site/708

https://dev.to/aws-builders/using-dynamodb-on-spring-boot-featkotlin-3674

https://refactorfirst.com/using-dynamodb-with-spring-boot

https://techfox.tistory.com/45

https://velog.io/@godol811/Spring Boot에서-DynamoDB-aws-java-sdk-v2-Test-code를-이용해서-적용하기

전체 예제 Git 링크