Table creation on read or write?

[joelbandi]

Hi team,

It seems like dynamoid is creating tables on the first write operation to that table but it doesn't do the same for the read operation. In our use case, it so happens that we have a find_first_or_create as the first operation in our rails app

That means we are going to be running into Aws::DynamoDB::Errors::ResourceNotFoundException (Cannot do operations on a non-existent table) everytime we add a new table with read operation first or everytime we run the build through CI (we use localstack to simulate our aws infrastructure in test).

irb(main):004:0> User.find('[email protected]') # User is a dynamoid document
describe_table | Request "{\"TableName\":\"screenings_users_test\"}" | Response "{\"message\": \"Cannot do operations on a non-existent table\", \"__type\": \"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\"}"
Traceback (most recent call last):
        2: from (irb):4
        1: from (irb):4:in `rescue in irb_binding'
Aws::DynamoDB::Errors::ResourceNotFoundException (Cannot do operations on a non-existent table)


irb(main):005:0> Aws::Dynamo::User.create(email: '[email protected]')

list_tables | Request "{}" | Response "{\"TableNames\":[]}"
(18.45 ms) LIST TABLES
(26.75 ms) CACHE TABLES
Creating screenings_users_test table. This could take a while.
create_table | Request "{\"TableName\":\"screenings_users_test\",\"KeySchema\":[{\"AttributeName\":\"email\",\"KeyType\":\"HASH\"}],\"AttributeDefinitions\":[{\"AttributeName\":\"email\",\"AttributeType\":\"S\"}],\"BillingMode\":\"PROVISIONED\",\"ProvisionedThroughput\":{\"ReadCapacityUnits\":100,\"WriteCapacityUnits\":20}}" | Response "{\"TableDescription\":{\"AttributeDefinitions\":[{\"AttributeName\":\"email\",\"AttributeType\":\"S\"}],\"TableName\":\"screenings_users_test\",\"KeySchema\":[{\"AttributeName\":\"email\",\"KeyType\":\"HASH\"}],\"TableStatus\":\"ACTIVE\",\"CreationDateTime\":1588788636.798,\"ProvisionedThroughput\":{\"LastIncreaseDateTime\":0.000,\"LastDecreaseDateTime\":0.000,\"NumberOfDecreasesToday\":0,\"ReadCapacityUnits\":100,\"WriteCapacityUnits\":20},\"TableSizeBytes\":0,\"ItemCount\":0,\"TableArn\": \"arn:aws:dynamodb:us-east-1:000000000000:table/screenings_users_test\",\"BillingModeSummary\":{\"BillingMode\":\"PROVISIONED\",\"LastUpdateToPayPerRequestDateTime\":0.000}}}"
(354.18 ms) CREATE TABLE
put_item | Request "{\"TableName\":\"screenings_users_test\",\"Item\":{\"email\":{\"S\":\"[email protected]\"},\"last_login\":{\"S\":\"2020-05-06T18:10:36+00:00\"}},\"Expected\":{\"email\":{\"Exists\":false}}}" | Response "{\"Attributes\": {\"email\": {\"S\": \"[email protected]\"}, \"last_login\": {\"S\": \"2020-05-06T18:10:36+00:00\"}}}"
(208.9 ms) PUT ITEM - ["screenings_users_test", {:email=>"[email protected]", :last_login=>"2020-05-06T18:10:36+00:00"}, {}]

Is this by design? Is there a recommended way to handle this issue currently?

[andrykonchin]

In the test environment the recommended way it to manually create all the missing tables with create_table method:

Dynamoid.included_models.each { |m| m.create_table(sync: true) }

See https://github.com/Dynamoid/dynamoid#test-environment for details

[joelbandi]

Are there plans to auto-create tables on reads in the future?

[andrykonchin]

I haven't considered this feature and have no strong opinion about it.

Implicit table creation can be very confusing especially in production environment. Implicitness can make things more complex and difficult. On the other hand it's really handy in development and test environments.

It seems reasonable to me to create tables implicitly at first "read" operation like find or where but there should be a configuration option to disable this implicit behavior (for both "read" and "write" operations) at all e.g. for production environment.

[joelbandi]

We already have some degree of implicitness that comes with table creation on first writes. IMO, it makes things more complex and confusing with partial implicitness.
On the other hand, I agree it'd be nice for us to set a configurable parameter to control implicit table creation. I'll try to create a PR if that's okay.

[andrykonchin]

Yeah, the PR is welcome.