bug: ASLParserException on CreateStateMachine operation if 'Map' state contains 'MaxConcurrencyPath' field

[attila]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Hello there, it looks like state machines fail to create when they contain a Map state with "MaxConcurrencyPath" defined. The error thrown is:

An error occurred (InvalidDefinition) when calling the CreateStateMachine operation: ASLParserException line 1:138, at "MaxConcurrencyPath", mismatched input '"MaxConcurrencyPath"' expecting {'"Comment"', '"Type"', '"Choices"', '"Default"', '"Branches"', '"SecondsPath"', '"Seconds"', '"TimestampPath"', '"Timestamp"', '"TimeoutSeconds"', '"TimeoutSecondsPath"', '"HeartbeatSeconds"', '"HeartbeatSecondsPath"', '"ItemProcessor"', '"Iterator"', '"ItemSelector"', '"MaxConcurrency"', '"Resource"', '"InputPath"', '"OutputPath"', '"ItemsPath"', '"ResultPath"', '"Result"', '"Parameters"', '"ResultSelector"', '"ItemReader"', '"Next"', '"End"', '"Cause"', '"CausePath"', '"Error"', '"ErrorPath"', '"Retry"', '"Catch"'}

Expected Behavior

Step function created successfully. See the option documented at https://docs.aws.amazon.com/step-functions/latest/dg/concepts-asl-use-map-state-inline.html#map-state-inline-additional-fields

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

docker compose up

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

awslocal stepfunctions create-state-machine --name 'example' --role-arn 'arn:aws:iam::000000000000:role/stepfunctions-role' --definition '{"StartAt":"MapState","States":{"MapState":{"Type":"Map","Iterator":{"StartAt":"Iterate","States":{"Iterate":{"Type":"Pass","End":true}}},"MaxConcurrencyPath":"$.Some.Path"}}}'

I inlined the definition in the example but here it is for better readability:

{
  "StartAt": "MapState",
  "States": {
    "MapState": {
      "Type": "Map",
      "Iterator": {
        "StartAt": "Iterate",
        "States": {
          "Iterate": {
            "Type": "Pass",
            "End": true
          }
        }
      },
      "MaxConcurrencyPath": "$.Some.Path"
    }
  }
}

Environment

- OS: MacOS 14
- LocalStack: 3.3.0

Anything else?

No response

[MEPalma]

@attila thank you for taking the time to compile this thorough report. I was able to replicate the behaviour. We recently merged some changes that aim to address this issue. These changes are scheduled to be included in the next nightly release too. I would be grateful if you could test the new build at your earliest convenience and provide feedback on whether it resolves the problem you encountered. Thank you once again for bringing this issue forward.

[attila]

@MEPalma thank you very much for looking into this.

I have tested the feature in the nightly build (2023-04-23) where I looked into the capability of creating state machines with the definition and also to verify that they work as intended. I achieved partial success in this, see details below.

I could create a state machine with the following definition:

{
  "StartAt": "MapState",
  "States": {
    "MapState": {
      "Type": "Map",
      "ItemsPath": "$.Items",
      "Iterator": {
        "StartAt": "Wait",
        "States": {
          "Wait": {
            "Type": "Wait",
            "Seconds": 5,
            "Next": "Proceed"
          },
          "Proceed": {
            "Type": "Pass",
            "End": true
          }
        }
      },
      "MaxConcurrencyPath": "$.MaxConcurrency",
      "End": true
    }
  }
}

Invoking the state machine with the following inputs verified that maximum concurrency was observed (by checking the "startDate" and "stopDate" of the executions)

{
  "MaxConcurrency": 5,
  "Items": [1,2,3,4,5,6,7,8,9,10]
}

However, when I specified the value of max concurrency as a string, the service did not cast it to a number like it does in the real AWS service, but failed with the following:

Exception=FailureEventException, Error=States.Runtime, Details={"executionFailedEventDetails": {"error": "States.Runtime", "cause": "The MaxConcurrencyPath field refers to value \"5\" which is not a valid integer: $.MaxConcurrency"}} at '(MaxConcurrencyPath| {'max_concurrency_value': 0, 'max_concurrency_path': '$.MaxConcurrency'}'

I think it's important to retain this capability because of a lack of string to number casting intrinsic functions available.

We do have a use case where the maximum concurrency would be set from a DynamoDB record attribute.
Example: given that DynamoDB returns these records as DynamoDB JSON, (i.e. { "max_concurrency": 5 } becomes { "max_concurrency": { "N": "5" }) and if I want to pick up the value as "MaxConcurrencyPath": "$.MyDynamoDBResult.Attributes.max_concurrency.N" it is impossible.