[Spring Boot] DynamoDB
Spring BootAmazon DynamoDB
완전 관리형 NoSQL Database(비관계형 데이터베이스) Service
Amazon Web Services에서만 사용 가능한 Serverless Instance로 서버를 직접 관리할 필요가 없다.
데이터를 Key-Value 구조로 저장하며 JSON을 지원한다.
스키마가 없기 때문에 테이블을 생성할 때 기본 키 외에 추가 속성에 대해서는 정의하지 않는다.
데이터 유형이 String, Number, Binary로 제한되어 있다.
가변 요금제로 데이터 읽기, 쓰기, 저장 등 사용량에 따라 요금이 부과된다.
백업, 배포, 확장이 편리하다.
Spring Boot와 DynamoDB
Spring Data에서 공식 지원하지 않는다. (링크)
@Transaction을 사용할 수 없다.
Git Library
- boostchicken: 3번째 분기된 라이브러리. 2020년 이후 업데이트되지 않고 있다.
- derjust: 2.2부터 지원되지 않는다.
- 최신 라이브러리?: boostchicken Issue(링크)에 따르면 awspring이 언급되지만 확실하지 않음
Docker localstack
local에서 테스트할 수 있는 AWS 클라우드 환경을 제공한다.
AWS 서비스를 대부분 지원한다. (AWS Service Feature Coverage)
docker-compose.yml (example link)
yml
version: "3"
services:
aws:
container_name: exam-aws
image: localstack/localstack
platform: linux/amd64
ports:
- "4566:4566" # LocalStack Gateway
- "4510-4559:4510-4559" # external services port range
environment:
- SERVICE=dynamodb # 사용할 서비스
- DEBUG=1
- DOCKER_HOST=unix:///var/run/docker.sock
# Container 내부에서 aws-cli를 사용하기 위해 설정. awslocal 사용 시 미지정
- AWS_ACCESS_KEY_ID=dev
- AWS_SECRET_ACCESS_KEY=dev
- AWS_DEFAULT_REGION=ap-northeast-2
volumes:
- "./localstack/data:/var/lib/localstack"
- "./localstack/init/ready.d:/etc/localstack/init/ready.d"
restart: unless-stopped
create_dynamodb.sh (./localstack/init/ready.d)
text
#!/bin/bash
echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name ad_event \
--attribute-definitions \
AttributeName=id,AttributeType=S \
--key-schema \
AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)
echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name button_event \
--attribute-definitions \
AttributeName=id,AttributeType=N \
AttributeName=username,AttributeType=S \
--key-schema \
AttributeName=id,KeyType=HASH \
AttributeName=username,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)
echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name click_event \
--attribute-definitions \
AttributeName=id,AttributeType=N \
AttributeName=reg_dtm,AttributeType=S \
--key-schema \
AttributeName=id,KeyType=HASH \
AttributeName=reg_dtm,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)
echo $(aws dynamodb create-table \
--endpoint-url http://localhost:4566 \
--table-name scroll_event \
--attribute-definitions \
AttributeName=id,AttributeType=S \
--key-schema \
AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5)
Spring Boot 방법 1. GIT Library boostchicken
build.gradle.kts
kotlin
implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.533")
implementation("io.github.boostchicken:spring-data-dynamodb:5.2.5")
Config
kotlin
import com.amazonaws.auth.AWSCredentialsProvider
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain
import com.amazonaws.client.builder.AwsClientBuilder
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig
import org.socialsignin.spring.data.dynamodb.repository.config.EnableDynamoDBRepositories
import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.context.annotation.Primary
@Configuration
@EnableDynamoDBRepositories(basePackages = ["com.example.tracker"])
class DynamoDBConfig(
@Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
@Value("\${cloud.aws.region.static}") private val region: String
) {
@Bean
@Primary
fun dynamoDBMapper(amazonDynamoDB: AmazonDynamoDB, dynamoDBMapperConfig: DynamoDBMapperConfig): DynamoDBMapper {
return DynamoDBMapper(amazonDynamoDB, dynamoDBMapperConfig)
}
@Bean
@Primary
fun awsCredentialsProvider(): AWSCredentialsProvider {
return DefaultAWSCredentialsProviderChain.getInstance() // 자격 증명을 환경변수, 프로필 파일(~/.aws/credentials) 순으로 탐색
}
@Bean
@Primary
fun dynamoDBMapperConfig(): DynamoDBMapperConfig {
return DynamoDBMapperConfig.DEFAULT
}
@Bean
fun amazonDynamoDB(): AmazonDynamoDB {
return AmazonDynamoDBClientBuilder
.standard()
.withEndpointConfiguration(AwsClientBuilder.EndpointConfiguration(endpoint, region))
.withCredentials(awsCredentialsProvider())
.build()
}
}
Domain
kotlin
import com.amazonaws.services.dynamodbv2.datamodeling.*
import com.example.tracker.common.converter.DynamoDbLocalDateTimeConverter
import java.time.LocalDateTime
import java.util.*
/**
* required: var, default value
*/
@DynamoDBTable(tableName = "ad_event")
data class AdEvent(
@DynamoDBHashKey
@DynamoDBAutoGeneratedKey
var id: String = UUID.randomUUID().toString(),
@DynamoDBAttribute(attributeName = "session_key")
var sessionKey: String = "",
@DynamoDBAttribute(attributeName = "reg_dtm")
@DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
var regDtm: LocalDateTime = LocalDateTime.now()
)
데이터 타입 변환 설정
DynamoDB 속성 타입은 String, Number, Binary만 존재하기 때문에 날짜로 변환하기 위한 Converter가 필요하다
kotlin
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter
import java.time.LocalDateTime
class DynamoDbLocalDateTimeConverter : DynamoDBTypeConverter<String, LocalDateTime> {
override fun convert(date: LocalDateTime?): String? {
return date?.toString()
}
override fun unconvert(value: String?): LocalDateTime? {
return value?.let { LocalDateTime.parse(it) }
}
}
Repository
kotlin
import com.example.tracker.ad.domain.AdEvent
import org.socialsignin.spring.data.dynamodb.repository.EnableScan
import org.springframework.data.repository.CrudRepository
import java.time.LocalDateTime
@EnableScan
interface AdEventRepository : CrudRepository<AdEvent, String> {
fun findByRegDtmGreaterThan(regDtm: LocalDateTime): List<AdEvent>
}
Service
kotlin
@Service
class AdEventService(
private val adEventRepository: AdEventRepository
) {
fun save(sessionKey: String) {
adEventRepository.save(
AdEvent(
sessionKey = sessionKey
)
)
}
fun findAll(): List<AdEvent> {
return adEventRepository.findAll().toList()
}
fun findAdEvent(regDtm: LocalDateTime): List<AdEvent> {
return adEventRepository.findByRegDtmGreaterThan(regDtm)
}
}
Spring Boot 방법 2. GIT Library, Primary Key
build.gradle.kts (위 예시와 동일)
kotlin
implementation("com.amazonaws:aws-java-sdk-dynamodb:1.12.533")
implementation("io.github.boostchicken:spring-data-dynamodb:5.2.5")
Config
text
방법 1과 동일
Domain
kotlin
/**
* DynamoDB는 Schema가 없기 때문에 기본값을 NULL로 하는게 안전
*/
@DynamoDBTable(tableName = "click_event")
data class ClickEvent(
@get:DynamoDBHashKey
var id: Long? = null,
@get:DynamoDBRangeKey(attributeName = "reg_dtm")
@DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
var regDtm: LocalDateTime? = null,
@DynamoDBAttribute(attributeName = "json_data")
var jsonData: String? = null
) {
// Spring Boot 3 (jakarta) 대응이 아직 안되어 있다.
@org.springframework.data.annotation.Id
private var key: PrimaryKey? = null
get() {
return PrimaryKey(id, regDtm)
}
}
Domain Key
kotlin
data class PrimaryKey(
@DynamoDBHashKey
var id: Long? = null,
@DynamoDBRangeKey(attributeName = "reg_dtm")
@DynamoDBTypeConverted(converter = DynamoDbLocalDateTimeConverter::class)
var regDtm: LocalDateTime? = null
)
Repository
kotlin
package com.example.tracker.click.repository
import com.example.tracker.click.domain.ClickEvent
import com.example.tracker.common.domain.PrimaryKey
import org.socialsignin.spring.data.dynamodb.repository.EnableScan
import org.springframework.data.repository.CrudRepository
@EnableScan
interface ClickEventRepository : CrudRepository<ClickEvent, PrimaryKey> {
fun findByKey(key: PrimaryKey): List<ClickEvent>
}
Service
kotlin
@Service
class ClickEventService(
private val clickEventRepository: ClickEventRepository
) {
fun save() {
clickEventRepository.save(
ClickEvent(
id = 8L,
regDtm = LocalDateTime.now(),
jsonData = Gson().toJson(JsonData("abcd"))
)
)
}
fun findAll(): List<ClickEvent> {
return clickEventRepository.findAll().toList()
}
fun findByKey(key: PrimaryKey): List<ClickEvent> {
return clickEventRepository.findByKey(key)
}
}
Spring Boot 방법 3. AWS SDK Kotlin
build.gradle.kts
kotlin
implementation("aws.sdk.kotlin:dynamodb-jvm:0.21.4-beta")
implementation("com.squareup.okhttp3:okhttp:5.0.0-alpha.11")
Client Provider
kotlin
@Component
class DynamoDbClientProvider(
@Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
@Value("\${cloud.aws.region.static}") private val regions: String
) {
fun client() = DynamoDbClient {
region = regions
endpointUrl = Url.parse(endpoint)
credentialsProvider = EnvironmentCredentialsProvider()
}
}
Domain
kotlin
import java.time.LocalDateTime
data class ClickEvent(
var id: Long? = null,
val regDtm: LocalDateTime? = null,
val jsonData: String? = null
)
Repository
kotlin
import aws.sdk.kotlin.services.dynamodb.model.AttributeValue
import aws.sdk.kotlin.services.dynamodb.model.QueryRequest
import com.example.tracker.click.domain.ClickEvent
import com.example.tracker.common.provider.DynamoDbClientProvider
import kotlinx.coroutines.runBlocking
import org.springframework.stereotype.Repository
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
@Repository
class ClickEventRepository(
private val clientProvider: DynamoDbClientProvider
) {
companion object {
const val TABLE_NAME = "click_event"
}
fun findByRegDtmBetween(
partitionVal: Long,
startDtm: LocalDateTime,
endDtm: LocalDateTime
): List<ClickEvent> {
val list = mutableListOf<ClickEvent>()
runBlocking {
val attr = mutableMapOf<String, AttributeValue>()
attr[":a"] = AttributeValue.N(partitionVal.toString())
attr[":t1"] = AttributeValue.S(startDtm.toString())
attr[":t2"] = AttributeValue.S(endDtm.toString())
val request = QueryRequest {
tableName = TABLE_NAME
keyConditionExpression = "id = :a AND reg_dtm BETWEEN :t1 AND :t2"
expressionAttributeValues = attr
}
clientProvider.client().use { ddb ->
ddb.query(request).items?.forEach {
val id = it.getValue("id").asN()
val regDtm = it.getValue("reg_dtm").asS()
val jsonData = it.getValue("json_data").asS()
list.add(
ClickEvent(
id = id.toLong(),
regDtm = LocalDateTime.parse(regDtm, DateTimeFormatter.ISO_DATE_TIME),
jsonData = jsonData
)
)
}
}
}
return list
}
}
Service
kotlin
@Service
class ClickEventService(
private val clickEventRepository: ClickEventRepository
) {
fun findByRegDtmBetween(
partitionVal: Long,
startDtm: LocalDateTime,
endDtm: LocalDateTime
): List<ClickEvent> {
return clickEventRepository.findByRegDtmBetween(partitionVal, startDtm, endDtm)
}
}
Spring Boot 방법 4. AWS SDK Kotlin 2
build.gradle.kts
kotlin
implementation("aws.sdk.kotlin:dynamodb-jvm:0.21.4-beta")
implementation("com.squareup.okhttp3:okhttp:5.0.0-alpha.11")
service
kotlin
import aws.sdk.kotlin.runtime.auth.credentials.EnvironmentCredentialsProvider
import aws.sdk.kotlin.services.dynamodb.DynamoDbClient
import aws.sdk.kotlin.services.dynamodb.model.*
import aws.smithy.kotlin.runtime.net.Url
import kotlinx.coroutines.runBlocking
import org.springframework.beans.factory.annotation.Value
import org.springframework.stereotype.Service
@Service
class ScrollEventService(
@Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
@Value("\${cloud.aws.region.static}") private val regions: String
) {
companion object {
const val TABLE_NAME = "scroll_event"
}
fun getAll() {
runBlocking {
val request = ScanRequest {
tableName = "scroll_event"
}
getClient().use { ddb ->
val response = ddb.scan(request)
response.items?.forEach { item ->
item.keys.forEach { key ->
println("key name: $key / value: ${item[key]}")
}
}
}
}
}
fun putItem(
key: String,
keyVal: String,
columnKey1: String,
columnKey1Val: String
) {
runBlocking {
val itemValues = mutableMapOf<String, AttributeValue>()
itemValues[key] = AttributeValue.S(keyVal)
itemValues[columnKey1] = AttributeValue.S(columnKey1Val)
val request = PutItemRequest {
tableName = TABLE_NAME
item = itemValues
}
getClient().use { ddb ->
ddb.putItem(request)
println("put item: $TABLE_NAME / key val: $keyVal")
}
}
}
/**
* @param keyName = Required key
*/
fun getItem(keyName: String, keyVal: String) {
runBlocking {
val keyToGet = mutableMapOf<String, AttributeValue>()
keyToGet[keyName] = AttributeValue.S(keyVal)
val request = GetItemRequest {
key = keyToGet
tableName = TABLE_NAME
}
getClient().use { ddb ->
val returnedItem = ddb.getItem(request)
val numberMap = returnedItem.item
numberMap?.forEach { key ->
println("key name: ${key.key} / value: ${key.value}")
}
}
}
}
fun keyConditionExpressionsForQuery(
partitionAlias: String,
partitionKeyName: String,
partitionKeyVal: String
) {
runBlocking {
val attrValues = mutableMapOf<String, AttributeValue>()
attrValues[":$partitionAlias"] = AttributeValue.S(partitionKeyVal)
val request = QueryRequest {
tableName = TABLE_NAME
keyConditionExpression = "$partitionKeyName = :$partitionAlias"
expressionAttributeValues = attrValues
}
getClient().use { ddb ->
val response = ddb.query(request)
println("count: ${response.count}")
response.items?.forEach { item ->
item.forEach { map ->
println("key: ${map.key} / value: ${map.value}")
}
println("--------------------")
}
}
}
}
fun filterForQuery(
filterKeyAlias: String,
filterKey: String,
filterKeyValAlias: String,
filterKeyVal: String,
partitionAlias: String,
partitionKeyName: String,
partitionKeyVal: String,
) {
runBlocking {
val attrNameAlias = mutableMapOf<String, String>()
attrNameAlias["#${filterKeyAlias}"] = filterKey
val attrValues = mutableMapOf<String, AttributeValue>()
attrValues[":$partitionAlias"] = AttributeValue.S(partitionKeyVal)
attrValues[":$filterKeyValAlias"] = AttributeValue.S(filterKeyVal)
val request = QueryRequest {
tableName = TABLE_NAME
keyConditionExpression = "$partitionKeyName = :$partitionAlias"
filterExpression = "#${filterKeyAlias} = :$filterKeyValAlias"
expressionAttributeNames = attrNameAlias
expressionAttributeValues = attrValues
}
getClient().use { ddb ->
val response = ddb.query(request)
println("count: ${response.count}")
response.items?.forEach { item ->
item.forEach { map ->
println("key: ${map.key} / value: ${map.value}")
}
println("--------------------")
}
}
}
}
private fun getClient() = DynamoDbClient {
region = regions
credentialsProvider = EnvironmentCredentialsProvider()
endpointUrl = Url.parse(endpoint)
}
}
Spring Boot 방법 5. AWS SDK JAVA
build.gradle.kts
kotlin
implementation("software.amazon.awssdk:dynamodb-enhanced")
implementation("io.reactivex.rxjava2:rxjava:2.2.21")
Config
kotlin
import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedAsyncClient
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient
import java.net.URI
@Configuration
class DynamoDBConfig(
@Value("\${cloud.aws.dynamodb.endpoint}") private val endpoint: String,
@Value("\${cloud.aws.region.static}") private val region: String,
@Value("\${cloud.aws.access-key-id}") private val accessKeyId: String,
@Value("\${cloud.aws.secret-access-key}") private val secretAccessKeyId: String,
) {
@Bean
fun dynamoDbAsyncClient(): DynamoDbAsyncClient {
return DynamoDbAsyncClient.builder()
.region(Region.of(region))
.endpointOverride(URI.create(endpoint))
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(
accessKeyId,
secretAccessKeyId
)
)
)
.build()
}
@Bean
fun dynamoDbEnhancedAsyncClient(): DynamoDbEnhancedAsyncClient {
return DynamoDbEnhancedAsyncClient.builder()
.dynamoDbClient(dynamoDbAsyncClient())
.build()
}
}
Domain
kotlin
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.LocalDateTimeAttributeConverter
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.*
import java.time.LocalDateTime
@DynamoDbBean
data class ButtonEvent(
@get:DynamoDbPartitionKey
var id: Long? = null,
@get:DynamoDbSortKey
var username: String? = null,
@get:DynamoDbAttribute("button_id")
var buttonId: Long? = null,
@get:DynamoDbAttribute("reg_dtm")
@get:DynamoDbConvertedBy(LocalDateTimeAttributeConverter::class)
var regDtm: LocalDateTime? = null
)
Repository
kotlin
import com.example.tracker.btn.domain.ButtonEvent
import io.reactivex.Flowable
import org.springframework.stereotype.Repository
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedAsyncClient
import software.amazon.awssdk.enhanced.dynamodb.Expression
import software.amazon.awssdk.enhanced.dynamodb.Key
import software.amazon.awssdk.enhanced.dynamodb.TableSchema
import software.amazon.awssdk.enhanced.dynamodb.model.QueryConditional
import software.amazon.awssdk.services.dynamodb.model.AttributeValue
import java.time.LocalDateTime
import java.util.concurrent.CompletableFuture
@Repository
class ButtonEventRepository(
private val client: DynamoDbEnhancedAsyncClient
) {
companion object {
private const val TABLE_NAME = "button_event"
private val tableSchema = TableSchema.fromBean(ButtonEvent::class.java)
}
private val table = client.table(TABLE_NAME, tableSchema)
fun save(buttonEvent: ButtonEvent) {
table.putItem(buttonEvent).get()
}
fun findByKey(partitionVal: Long, sortVal: String): ButtonEvent? {
val key = Key.builder().partitionValue(partitionVal).sortValue(sortVal).build()
val result: CompletableFuture<ButtonEvent>? = table.getItem { it.key(key) }
return result?.get()
}
fun findByKeyAndRegDtm(
partitionVal: Long,
sortVal: String,
startDtm: LocalDateTime,
endDtm: LocalDateTime
): List<ButtonEvent> {
val values = mutableMapOf<String, AttributeValue>()
values[":start_dtm"] = AttributeValue.builder().s(startDtm.toString()).build()
values[":end_dtm"] = AttributeValue.builder().s(endDtm.toString()).build()
val expression = Expression.builder()
.expression("reg_dtm >= :start_dtm AND reg_dtm <= :end_dtm")
.expressionValues(values)
.build()
val queryConditional = QueryConditional
.keyEqualTo(Key.builder().partitionValue(partitionVal).sortValue(sortVal).build())
val result = table.query { r ->
r.queryConditional(queryConditional)
.filterExpression(expression)
}.items()
return Flowable.fromPublisher(result).toList().blockingGet()
}
}
Service
kotlin
@Service
class ButtonEventService(
private val buttonEventRepository: ButtonEventRepository
) {
fun save(buttonEvent: ButtonEvent) {
buttonEventRepository.save(buttonEvent)
}
fun findByKey(partitionVal: Long, sortVal: String): ButtonEvent? {
return buttonEventRepository.findByKey(partitionVal, sortVal)
}
fun findByKeyAndRegDtm(
partitionVal: Long,
sortVal: String,
startDtm: LocalDateTime,
endDtm: LocalDateTime
): List<ButtonEvent> {
return buttonEventRepository.findByKeyAndRegDtm(partitionVal, sortVal, startDtm, endDtm)
}
}
Reference
https://howtodoinjava.com/spring-boot/access-dynamodb-with-spring
https://betterprogramming.pub/aws-java-sdk-v2-dynamodb-enhanced-client-with-kotlin-spring-boot-application-f880c74193a2
https://kouzie.github.io/aws/aws-dynamodb/#%EA%B8%B0%EB%B3%B8-%ED%82%A4
https://jsonobject.tistory.com/599
https://wedul.site/708
https://dev.to/aws-builders/using-dynamodb-on-spring-boot-featkotlin-3674
https://refactorfirst.com/using-dynamodb-with-spring-boot
https://techfox.tistory.com/45
https://velog.io/@godol811/Spring Boot에서-DynamoDB-aws-java-sdk-v2-Test-code를-이용해서-적용하기
전체 예제 Git 링크