Dynamoid unable to read datetime field stored as Number after version update from 2.2.x to 3.7.x

[Ashish-Jaiswal98]

The datetime fields are not getting converted in Dynamodb model class to Datatime object. Instead they are returning nil object.
Example of use -
class Test
include Dynamoid::Document
table :name => :Table, :key => :Key
field :timestamp, :datetime
end

Field timestamp returns nil object now. Before it used to return Datatime object with version 2.2.x.
This issue I am facing for Magic fields also like created_at, updated_at. Even though they are stored as Number in DDB. They are being returned as nil object.

[andrykonchin]

Could you please provide the exact steps to reproduce the issue?

---

Unfortunately, I didn't manage to reproduce the issue on my own.

Model declaration:

class TestTimestamp
  include Dynamoid::Document
  field :timestamp, :datetime
end

Create an item. Environment: Dynamoid v2.2, Rails 5, ruby 2.2

require 'pp'

obj = TestTimestamp.create(timestamp: Time.now)
pp Dynamoid.adapter.scan(TestTimestamp.table_name).to_a

Result - store timestamp/created_at/updated_at as BigDecimal:

[{:created_at=>#<BigDecimal:7facdb54db90,'0.1624397248 328982E10',27(27)>,
  :id=>"8a8de688-448a-43f7-8507-857455f081dc",
  :updated_at=>#<BigDecimal:7facdb54d8c0,'0.1624397248 329424E10',27(27)>,
  :timestamp=>#<BigDecimal:7facdb54d708,'0.1624397248 133721E10',27(27)>}]

Read items. Environment: Dynamoid v3.7, Rails 6, Ruby 3.0

objs = TestTimestamp.all.to_a

=> [#<TestTimestamp:0x00007fdae64de4f0 @new_record=false, @attributes={:created_at=>Tue, 22 Jun 2021 21:27:28 +0000, :updated_at=>Tue, 22 Jun 2021 21:27:28 +0000, :id=>"8a8d...

objs[0].attributes
=> {:created_at=>Tue, 22 Jun 2021 21:27:28 +0000, :updated_at=>Tue, 22 Jun 2021 21:27:28 +0000, :id=>"8a8de688-448a-43f7-8507-857455f081dc", :timestamp=>Tue, 22 Jun 2021 21:27:28 +0000}

objs[0].timestamp
=> Tue, 22 Jun 2021 21:27:28 +0000

As you can see datetime attributes have correct values.

[Ashish-Jaiswal98]

The creation part is working fine for me. And when I use the .all API (for reading) then i am able to get the datetime fields.
In my code I am using .query API to query my table and using TestTimestamp.new(item) to cast the result to the data model. This cast is not working properly using .new for datetime fields.

Ex of my query logic :

    rows = []
    result = Aws::DynamoDB::Table.new("TestTimestamp").query({
      attributes_to_get: ["id","updated_at","created_at", "timestamp"],
      key_conditions: {
          "id" => {
              attribute_value_list: ["1"],
              comparison_operator: "EQ"
            }
        }
    })

    result.items.each do |item|
      Rails.logger.info item                                 # Here I get datetime fields as BigDecimal
      rows << TestTimestamp.new(item)
    end

    Rails.logger.info rows[0].attributes.           # Here I get nil for datatime fields

[andrykonchin]

Ah, I see. It's an expected behaviour.

Type casting was introduced in the 3.0 version (changelog) and it prevents using raw values in some cases. Type casting for datetime field accepts only Strings and objects with to_datetime method.

[Ashish-Jaiswal98]

Is it possible to make some fix so that it can type cast datetime fields ?

[andrykonchin]

The current approach in typecasting is to copy Rails' behavior. As far as I know, Rails doesn't support type casting of Integer value for a DateTime attribute.

I don't have strong arguments against the proposed change but IMHO it makes sense if it's really needed in the community.

It can be implemented easily with monkey-patching of the proper type cast class.

[Ashish-Jaiswal98]

I think this is a valid fix to make as this was supported by previous versions of Dynamoid.
For now seems like I will have to go with the monkey-patching. Can you help me with the patch code as I am not very sure on how to implement it.

[andrykonchin]

You can patch type casting of datetime attributes in the following way:

module Dynamoid
  module TypeCasting
    module DateTimeTypeCastingWithNumericFormat
      def process(value)
        return Time.at(value.to_i) if value.respond_to?(:to_i)

        super
      end
    end

    DateTimeTypeCaster.send(:prepend, DateTimeTypeCastingWithNumericFormat)
  end
end

obj = TestTimestamp.new(timestamp: Time.now.to_i)
=> #<TestTimestamp:0x00007fca4a289e18 @new_record=true, @attributes={:timestamp=>2021-06-24 01:31:34 +0300}, @associations={}, @attributes_before_type_cast={:timestamp=>1624...

The only issue is that I use Time.at(value.to_i) so it returns a Time object in the system timezone. You may need to use Rails application timezone instead - Time.zone.at(value.to_i)