Missing blockid in azurite signature

[hashim-hivery]

Which service(blob, file, queue, table) does this issue concern?

Blob

Which version of the Azurite was used?

3.29.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

DockerHub -- mcr.microsoft.com/azure-storage/azurite@sha256:fc94c34a746aee451b7c3a46dda9d7e8844f4edaab5313862fd7ec04519fbbd0

What's the Node.js version?

v14.21.3

What problem was encountered?

Azurite seems to be omitting blockid query parameter from the signature calculation

Steps to reproduce the issue?

The first line in the below log content obtained from debug mode is the exact failing request.

The "string to sign" used by the server,

"PUT\n\n\n1536\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 07 Mar 2024 02:00:30 GMT\nx-ms-version:2023-11-03\n/devstoreaccount1/devstoreaccount1/<REDACTED>/test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv\ncomp:block

Is different from the "string to sign" used in my code:

PUT\n\n\n1536\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 07 Mar 2024 02:00:30 GMT\nx-ms-version:2023-11-03\n/devstoreaccount1/devstoreaccount1/<REDACTED>/test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv\nblockid:MDAwMDE=\ncomp:block

Log content:

2024-03-07T02:00:30.788Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://host.docker.internal/devstoreaccount1/<REDACTED>/test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv?comp=block&blockid=MDAwMDE= RequestHeaders:{"host":"host.docker.internal:10000","user-agent":"Go-http-client/1.1","content-length":"1536","authorization":"SharedKey devstoreaccount1:M0ljj0/4ED13d49Vwa8qWFHyPxAe1x8kEGFTi1x40dM=","x-ms-date":"Thu, 07 Mar 2024 02:00:30 GMT","x-ms-version":"2023-11-03","accept-encoding":"gzip"} ClientIP=192.168.65.1 Protocol=http HTTPVersion=1.1
2024-03-07T02:00:30.790Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=<REDACTED> Blob=test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv
2024-03-07T02:00:30.790Z 6780bd55-9555-409a-912f-45c9db86d928 verbose: DispatchMiddleware: Dispatching request...
2024-03-07T02:00:30.792Z 6780bd55-9555-409a-912f-45c9db86d928 info: DispatchMiddleware: Operation=BlockBlob_StageBlock
2024-03-07T02:00:30.792Z 6780bd55-9555-409a-912f-45c9db86d928 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-03-07T02:00:30.792Z 6780bd55-9555-409a-912f-45c9db86d928 info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-03-07T02:00:30.792Z 6780bd55-9555-409a-912f-45c9db86d928 debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-03-07T02:00:30.792Z 6780bd55-9555-409a-912f-45c9db86d928 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: <REDACTED>, blob: test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv
2024-03-07T02:00:30.794Z 6780bd55-9555-409a-912f-45c9db86d928 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container <REDACTED>
2024-03-07T02:00:30.794Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-03-07T02:00:30.795Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\n\n1536\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 07 Mar 2024 02:00:30 GMT\nx-ms-version:2023-11-03\n/devstoreaccount1/devstoreaccount1/<REDACTED>/test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv\ncomp:block"
2024-03-07T02:00:30.795Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:U+K340q5a90rr9KMumzIlp1+yY5eNr5hH1LUHTS8qek=
2024-03-07T02:00:30.795Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobSharedKeyAuthenticator:validate() Validation failed.
2024-03-07T02:00:30.795Z 6780bd55-9555-409a-912f-45c9db86d928 info: AccountSASAuthenticator:validate() Start validation against account Shared Access Signature pattern.
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: AccountSASAuthenticator:validate() Getting account properties...
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: AccountSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: <REDACTED>, blob: test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: AccountSASAuthenticator:validate() Got account properties successfully.
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: AccountSASAuthenticator:validate() Retrieved signature from URL parameter sig: undefined
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 info: AccountSASAuthenticator:validate() Failed to get valid account SAS values from request.
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 info: BlobSASAuthenticator:validate() Start validation against blob service Shared Access Signature pattern.
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: BlobSASAuthenticator:validate() Getting account properties...
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: BlobSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: <REDACTED>, blob: test_tenant/2024/03/07/uploads/02-00/<REDACTED>-csv
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: BlobSASAuthenticator:validate() Got account properties successfully.
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: BlobSASAuthenticator:validate() Retrieved signature from URL parameter sig: undefined
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 debug: BlobSASAuthenticator:validate() No signature found in request. Skip blob service SAS validation.
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.  ErrorHTTPStatusCode=403 ErrorHTTPStatusMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. ErrorHTTPHeaders={"x-ms-error-code":"AuthorizationFailure","x-ms-request-id":"6780bd55-9555-409a-912f-45c9db86d928"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>AuthorizationFailure</Code>\n  <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:6780bd55-9555-409a-912f-45c9db86d928\nTime:2024-03-07T02:00:30.796Z</Message>\n</Error>" ErrorStack="StorageError: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\n    at Function.getAuthorizationFailure (/opt/azurite/dist/src/blob/errors/StorageErrorFactory.js:137:16)\n    at /opt/azurite/dist/src/blob/middlewares/AuthenticationMiddlewareFactory.js:25:56\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)"
2024-03-07T02:00:30.796Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Set HTTP code: 403
2024-03-07T02:00:30.797Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Set HTTP status message: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.
2024-03-07T02:00:30.797Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=AuthorizationFailure
2024-03-07T02:00:30.797Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=6780bd55-9555-409a-912f-45c9db86d928
2024-03-07T02:00:30.797Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Set content type: application/xml
2024-03-07T02:00:30.798Z 6780bd55-9555-409a-912f-45c9db86d928 error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>AuthorizationFailure</Code>\n  <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:6780bd55-9555-409a-912f-45c9db86d928\nTime:2024-03-07T02:00:30.796Z</Message>\n</Error>"
2024-03-07T02:00:30.800Z 6780bd55-9555-409a-912f-45c9db86d928 info: EndMiddleware: End response. TotalTimeInMS=12 StatusCode=403 StatusMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. Headers={"server":"Azurite-Blob/3.29.0","x-ms-error-code":"AuthorizationFailure","x-ms-request-id":"6780bd55-9555-409a-912f-45c9db86d928","content-type":"application/xml"}

Have you found a mitigation/solution?

No

[hashim-hivery]

My code, as it stands, does work with the real azure blob storage, just doesn't work with azurite

[blueww]

@hashim-hivery

I can't repro this issue with latest Azurite.
In my test, the block ID will always be added to the StringTosign.

And per Azurite code here, you can see the Uri Query parameters will be added to StringTosign.

Azurite debug log in my test, you can see StringToSign contains blockid.

2024-03-14T03:54:49.285Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://127.0.0.1/devstoreaccount1/weitest/test.txt?comp=block&blockid=M1JpL09vNEtLajVUNG1ZY2Y3VTFOQT09MTIzNDU2NzgtMDAwMDIz&timeout=300 RequestHeaders:{"x-ms-client-request-id":"Azure-Storage-PowerShell-29e2590e-7d41-4f7b-88e2-51262206f2c4","user-agent":"AzurePowershell/v1.0.0 DataMovement/2.0.0.3 Azure-Storage/11.2.2 (.NET Core; Win32NT 10.0.22631.0)","x-ms-version":"2019-07-07","x-ms-date":"Thu, 14 Mar 2024 03:54:49 GMT","authorization":"SharedKey devstoreaccount1:QAREC9rk0mXO0WRuzfBDN24ghRwy088rA0HqGKxBgeY=","content-md5":"UeMgOhJLUI4E2FskZHFWcw==","host":"127.0.0.1:10000","content-length":"8388608"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2024-03-14T03:54:49.287Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=weitest Blob=test.txt
2024-03-14T03:54:49.287Z f64ff023-4e23-47a1-9452-9eca36e93d9a verbose: DispatchMiddleware: Dispatching request...
2024-03-14T03:54:49.289Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: DispatchMiddleware: Operation=BlockBlob_StageBlock
2024-03-14T03:54:49.290Z f64ff023-4e23-47a1-9452-9eca36e93d9a verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-03-14T03:54:49.291Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-03-14T03:54:49.291Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-03-14T03:54:49.292Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: weitest, blob: test.txt
2024-03-14T03:54:49.293Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container weitest
2024-03-14T03:54:49.294Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-03-14T03:54:49.294Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\n\n8388608\nUeMgOhJLUI4E2FskZHFWcw==\n\n\n\n\n\n\n\nx-ms-client-request-id:Azure-Storage-PowerShell-29e2590e-7d41-4f7b-88e2-51262206f2c4\nx-ms-date:Thu, 14 Mar 2024 03:54:49 GMT\nx-ms-version:2019-07-07\n/devstoreaccount1/devstoreaccount1/weitest/test.txt\nblockid:M1JpL09vNEtLajVUNG1ZY2Y3VTFOQT09MTIzNDU2NzgtMDAwMDIz\ncomp:block\ntimeout:300"
2024-03-14T03:54:49.295Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:QAREC9rk0mXO0WRuzfBDN24ghRwy088rA0HqGKxBgeY=
2024-03-14T03:54:49.295Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2024-03-14T03:54:49.296Z f64ff023-4e23-47a1-9452-9eca36e93d9a verbose: DeserializerMiddleware: Start deserializing...
2024-03-14T03:54:49.297Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: HandlerMiddleware: DeserializedParameters={"blockId":"M1JpL09vNEtLajVUNG1ZY2Y3VTFOQT09MTIzNDU2NzgtMDAwMDIz","options":{"timeout":300,"transactionalContentMD5":{"type":"Buffer","data":[81,227,32,58,18,75,80,142,4,216,91,36,100,113,86,115]},"requestId":"Azure-Storage-PowerShell-29e2590e-7d41-4f7b-88e2-51262206f2c4","leaseAccessConditions":{},"cpkInfo":{},"cpkScopeInfo":{}},"comp":"block","contentLength":8388608,"version":"2019-07-07","body":"ReadableStream"}
2024-03-14T03:54:49.297Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: OperationQueue.operate() Schedule incoming job e0bb303d-e61a-4eba-a520-765d3db0e68b
2024-03-14T03:54:49.297Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: OperationQueue:execute() Current runningConcurrency:3 maxConcurrency:50 operations.length:1
2024-03-14T03:54:49.298Z f64ff023-4e23-47a1-9452-9eca36e93d9a info: FSExtentStore:appendExtent() Select extent from idle location for extent append operation. LocationId:4 extentId:95ef8be7-c365-4d8a-adae-983adfb9d13a offset:16777216 MAX_EXTENT_SIZE:67108864 
2024-03-14T03:54:49.298Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: FSExtentStore:appendExtent() Get fd:7 for extent:95ef8be7-c365-4d8a-adae-983adfb9d13a from cache.
2024-03-14T03:54:49.298Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: FSExtentStore:appendExtent() Created write stream for fd:7
2024-03-14T03:54:49.299Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: FSExtentStore:appendExtent() Start writing to extent 95ef8be7-c365-4d8a-adae-983adfb9d13a
2024-03-14T03:54:49.299Z f64ff023-4e23-47a1-9452-9eca36e93d9a debug: FSExtentStore:streamPipe() Start piping data to write stream

[hashim-hivery]

@blueww Maybe there's some parsing problem somewhere, can you try with the same block id that I used?

[EmmaZhu]

Hi @hashim-hivery ,

I tried with the same block id you provided, it also works well.

2024-03-15T09:22:25.657Z 0aeecaef-052f-4754-a894-752c6a986a05 info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://127.0.0.1/devstoreaccount1/container171049454435304424/blob171049454440503068?comp=block&blockid=MDAwMDE%3D RequestHeaders:{"x-ms-version":"2023-08-03","content-length":"10","content-type":"application/octet-stream","accept":"application/xml","user-agent":"azsdk-js-storageblob/12.16.0 (NODE-VERSION v18.19.0; Windows_NT 10.0.22631)","x-ms-client-request-id":"e4147f27-41db-46ae-9312-a4a5deb3206f","x-ms-date":"Fri, 15 Mar 2024 09:22:25 GMT","authorization":"SharedKey devstoreaccount1:pFuvy+348iEQKLNgMbWZuRbL0v9Ug9BevGEgNIuNEM4=","host":"127.0.0.1:11000","connection":"close"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2024-03-15T09:22:25.658Z 0aeecaef-052f-4754-a894-752c6a986a05 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=container171049454435304424 Blob=blob171049454440503068
2024-03-15T09:22:25.659Z 0aeecaef-052f-4754-a894-752c6a986a05 verbose: DispatchMiddleware: Dispatching request...
2024-03-15T09:22:25.662Z 0aeecaef-052f-4754-a894-752c6a986a05 info: DispatchMiddleware: Operation=BlockBlob_StageBlock
2024-03-15T09:22:25.664Z 0aeecaef-052f-4754-a894-752c6a986a05 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-03-15T09:22:25.665Z 0aeecaef-052f-4754-a894-752c6a986a05 info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-03-15T09:22:25.665Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-03-15T09:22:25.665Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: container171049454435304424, blob: blob171049454440503068
2024-03-15T09:22:25.669Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container container171049454435304424
2024-03-15T09:22:25.670Z 0aeecaef-052f-4754-a894-752c6a986a05 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-03-15T09:22:25.671Z 0aeecaef-052f-4754-a894-752c6a986a05 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\n\n10\n\napplication/octet-stream\n\n\n\n\n\n\nx-ms-client-request-id:e4147f27-41db-46ae-9312-a4a5deb3206f\nx-ms-date:Fri, 15 Mar 2024 09:22:25 GMT\nx-ms-version:2023-08-03\n/devstoreaccount1/devstoreaccount1/container171049454435304424/blob171049454440503068\nblockid:MDAwMDE=\ncomp:block"
2024-03-15T09:22:25.671Z 0aeecaef-052f-4754-a894-752c6a986a05 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:pFuvy+348iEQKLNgMbWZuRbL0v9Ug9BevGEgNIuNEM4=
2024-03-15T09:22:25.672Z 0aeecaef-052f-4754-a894-752c6a986a05 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2024-03-15T09:22:25.672Z 0aeecaef-052f-4754-a894-752c6a986a05 verbose: DeserializerMiddleware: Start deserializing...
2024-03-15T09:22:25.673Z 0aeecaef-052f-4754-a894-752c6a986a05 info: HandlerMiddleware: DeserializedParameters={"blockId":"MDAwMDE=","options":{"requestId":"e4147f27-41db-46ae-9312-a4a5deb3206f","leaseAccessConditions":{},"cpkInfo":{},"cpkScopeInfo":{}},"comp":"block","contentLength":10,"version":"2023-08-03","body":"ReadableStream"}
2024-03-15T09:22:25.675Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue.operate() Schedule incoming job ff19fdb8-252b-4eee-aa64-edcb42ec4421
2024-03-15T09:22:25.676Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue:execute() Current runningConcurrency:0 maxConcurrency:10 operations.length:1
2024-03-15T09:22:25.677Z 0aeecaef-052f-4754-a894-752c6a986a05 info: FSExtentStore:appendExtent() Select extent from idle location for extent append operation. LocationId:1 extentId:342da729-4d82-4c3a-b7bd-0d8a20cfc073 offset:0 MAX_EXTENT_SIZE:67108864 
2024-03-15T09:22:25.677Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:appendExtent() Get fd:undefined for extent:342da729-4d82-4c3a-b7bd-0d8a20cfc073 from cache.
2024-03-15T09:22:25.679Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:appendExtent() Open file:__test_blob_extent__\342da729-4d82-4c3a-b7bd-0d8a20cfc073 for extent:342da729-4d82-4c3a-b7bd-0d8a20cfc073, get new fd:3
2024-03-15T09:22:25.681Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:appendExtent() Created write stream for fd:3
2024-03-15T09:22:25.681Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:appendExtent() Start writing to extent 342da729-4d82-4c3a-b7bd-0d8a20cfc073
2024-03-15T09:22:25.682Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:streamPipe() Start piping data to write stream
2024-03-15T09:22:25.683Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:streamPipe() Readable stream triggers close event, 10 bytes piped
2024-03-15T09:22:25.684Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:streamPipe() Invoke write stream end()
2024-03-15T09:22:25.684Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:streamPipe() Readable stream triggers close event, 10 bytes piped
2024-03-15T09:22:25.685Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:streamPipe() Writable stream triggers finish event, after 10 bytes piped. Flush data to fd:3.
2024-03-15T09:22:25.696Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:streamPipe() Flush data to fd:3 successfully. Resolve streamPipe().
2024-03-15T09:22:25.697Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:appendExtent() Write finish, start updating extent metadata. extent:{"id":"342da729-4d82-4c3a-b7bd-0d8a20cfc073","locationId":"test","path":"342da729-4d82-4c3a-b7bd-0d8a20cfc073","size":10,"lastModifiedInMS":1710494545697}
2024-03-15T09:22:25.698Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: FSExtentStore:appendExtent() Update extent metadata done. Resolve()
2024-03-15T09:22:25.698Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue.operate() Job ff19fdb8-252b-4eee-aa64-edcb42ec4421 completes callback, resolve.
2024-03-15T09:22:25.699Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue.operate() Schedule incoming job ee382f60-f4b9-44a3-a5aa-976da35db131
2024-03-15T09:22:25.699Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue:execute() Current runningConcurrency:0 maxConcurrency:100 operations.length:1
2024-03-15T09:22:25.699Z 0aeecaef-052f-4754-a894-752c6a986a05 verbose: FSExtentStore:readExtent() Creating read stream. LocationId:test extentId:342da729-4d82-4c3a-b7bd-0d8a20cfc073 path:__test_blob_extent__\342da729-4d82-4c3a-b7bd-0d8a20cfc073 offset:0 count:10 end:9
2024-03-15T09:22:25.700Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue.operate() Job ee382f60-f4b9-44a3-a5aa-976da35db131 completes callback, resolve.
2024-03-15T09:22:25.700Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue:execute() Current runningConcurrency:0 maxConcurrency:10 operations.length:0
2024-03-15T09:22:25.700Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue:execute() return. Operation.length === 0
2024-03-15T09:22:25.701Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue:execute() Current runningConcurrency:0 maxConcurrency:100 operations.length:0
2024-03-15T09:22:25.701Z 0aeecaef-052f-4754-a894-752c6a986a05 debug: OperationQueue:execute() return. Operation.length === 0
2024-03-15T09:22:25.705Z 0aeecaef-052f-4754-a894-752c6a986a05 verbose: SerializerMiddleware: Start serializing...
2024-03-15T09:22:25.707Z 0aeecaef-052f-4754-a894-752c6a986a05 info: EndMiddleware: End response. TotalTimeInMS=51 StatusCode=201 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.29.0","x-ms-client-request-id":"e4147f27-41db-46ae-9312-a4a5deb3206f","x-ms-request-id":"0aeecaef-052f-4754-a894-752c6a986a05","x-ms-version":"2024-02-04","date":"Fri, 15 Mar 2024 09:22:25 GMT","x-ms-request-server-encrypted":"true"}
2024-03-15T09:22:25.709Z 0aeecaef-052f-4754-a894-752c6a986a05 verbose: FSExtentStore:readExtent() Read stream closed. LocationId:test extentId:342da729-4d82-4c3a-b7bd-0d8a20cfc073 path:__test_blob_extent__\342da729-4d82-4c3a-b7bd-0d8a20cfc073 offset:0 count:10 end:9

Maybe you can share client side code you used, so we can try the same code to try to reproduce the issue?

Thanks
Emma

[EmmaZhu]

Hi @hashim-hivery ,

We found the root cause now.

The block id in your URL is like: blockid=MDAwMDE= , in my test case, it's blockid=MDAwMDE%3D. The SDK I used would encode character =.

From our document, the reserved characters in query parameter value should be encoded: https://learn.microsoft.com/en-us/graph/query-parameters?tabs=http#encoding-query-parameters. = is one of the reserved characters.

Azurite implementation meets the documentation's requirement.

Thanks
Emma