File ownership state is out of sync when doing multiple LspRename on unsaved buffers on multiple files

[lieryan]

Summary

After doing an LspRename, if the rename affects an unopened buffer, vim-lsp loses file synchronization with pylsp. So we start by renaming a variable in file moda.py, which imports a variable from modb.py; when we do the rename twice in a row, the second rename fails.

I've been reading through the logs and running debugger on the LSP (pylsp), I think the root cause of this issue seems te be vim-lsp and pylsp not agreeing who is controlling a file. When rope tries to read the modb.py from the workspace for the second rename operation, pylsp does not have the current state of the file, so pylsp reads the file from the filesystem which contains an older version of modb.py, but reads moda.py from the client as it is opened by the LSP client. Since modb.py already has unsaved changes from the previous rename command, modb.py should've been read from the client. I'm not familiar enough with the file synchronization mechanism in LSP, but I'm not quite sure who is supposed to own the file at this point.

Setup

1. Make sure vim is NOT configured to autosave files.

2. Setup a python venv, use the pylsp_rope rename plugin from pylsp-rope from this pull request. The builtin jedi_rename and rope_rename plugin should be disabled:

   $ python3 -m venv pylsp-venv
   $ source pylsp-venv/bin/activate
   $ pip install -e git+https://github.com/python-rope/pylsp-rope.git@lieryan-implement-rename#egg=pylsp-rope

3. Minimal ~/.vimrc:

   filetype on
   syntax on
   
   autocmd User lsp_setup call lsp#register_server({
             \   'name': 'pylsp',
             \   'cmd': {server_info->['pylsp', '-v', '--log-file', '/tmp/log.txt']},
             \   'allowlist': ['python'],
             \   'workspace_config': {
             \     'pylsp': {
             \       'plugins': {
             \         'jedi_rename': {
             \           'enabled': v:false,
             \         },
             \         'rope_rename': {
             \           'enabled': v:false,
             \         },
             \         'pylsp_rope': {
             \           'enabled': v:true,
             \           'rename': v:true,
             \         },
             \       },
             \     },
             \   },
             \ })

Reproduce

1. Create two files:

   # moda.py
   
   import modb
   
   print(modb.three)

   # modb.py
   three = 1

2. Open moda.py, point the cursor to three. :LspRename the variable three to four. This rename should successfully modify both moda.py and modb.py correctly in-buffer.

3. Without saving, quitting vim, or switching tabs, :LspRename the variable four to five. This would only modify moda.py, but not modb.py.

LSP server is python-lsp-server==1.10.0 with the pylsp_rope rename plugin from this pull request.

Client capabilities

{
  "method": "initialize",
  "params": {
    "rootUri": "file:///Users/lieryan/Projects/rope/pylsp-rope",
    "capabilities": {
      "workspace": {
        "workspaceFolders": false,
        "configuration": true,
        "symbol": {
          "dynamicRegistration": false
        },
        "applyEdit": true
      },
      "window": {
        "workDoneProgress": true
      },
      "textDocument": {
        "callHierarchy": {
          "dynamicRegistration": false
        },
        "rename": {
          "prepareSupport": true,
          "dynamicRegistration": false,
          "prepareSupportDefaultBehavior": 1
        },
        "codeAction": {
          "isPreferredSupport": true,
          "disabledSupport": true,
          "codeActionLiteralSupport": {
            "codeActionKind": {
              "valueSet": [
                "",
                "quickfix",
                "refactor",
                "refactor.extract",
                "refactor.inline",
                "refactor.rewrite",
                "source",
                "source.organizeImports"
              ]
            }
          },
          "dynamicRegistration": false
        },
        "completion": {
          "completionItem": {
            "snippetSupport": false,
            "resolveSupport": {
              "properties": [
                "additionalTextEdits"
              ]
            },
            "documentationFormat": [
              "markdown",
              "plaintext"
            ]
          },
          "dynamicRegistration": false,
          "completionItemKind": {
            "valueSet": [
              10,
              11,
              12,
              13,
              14,
              15,
              16,
              17,
              18,
              19,
              20,
              21,
              22,
              23,
              24,
              25,
              1,
              2,
              3,
              4,
              5,
              6,
              7,
              8,
              9
            ]
          }
        },
        "formatting": {
          "dynamicRegistration": false
        },
        "codeLens": {
          "dynamicRegistration": false
        },
        "inlayHint": {
          "dynamicRegistration": false
        },
        "hover": {
          "dynamicRegistration": false,
          "contentFormat": [
            "markdown",
            "plaintext"
          ]
        },
        "rangeFormatting": {
          "dynamicRegistration": false
        },
        "declaration": {
          "dynamicRegistration": false,
          "linkSupport": true
        },
        "references": {
          "dynamicRegistration": false
        },
        "typeHierarchy": {
          "dynamicRegistration": false
        },
        "foldingRange": {
          "rangeLimit": 5000,
          "dynamicRegistration": false,
          "lineFoldingOnly": true
        },
        "documentSymbol": {
          "symbolKind": {
            "valueSet": [
              10,
              11,
              12,
              13,
              14,
              15,
              16,
              17,
              18,
              19,
              20,
              21,
              22,
              23,
              24,
              25,
              26,
              1,
              2,
              3,
              4,
              5,
              6,
              7,
              8,
              9
            ]
          },
          "dynamicRegistration": false,
          "labelSupport": false,
          "hierarchicalDocumentSymbolSupport": false
        },
        "publishDiagnostics": {
          "relatedInformation": true
        },
        "synchronization": {
          "dynamicRegistration": false,
          "willSaveWaitUntil": false,
          "willSave": false,
          "didSave": true
        },
        "documentHighlight": {
          "dynamicRegistration": false
        },
        "implementation": {
          "dynamicRegistration": false,
          "linkSupport": true
        },
        "typeDefinition": {
          "dynamicRegistration": false,
          "linkSupport": true
        },
        "semanticTokens": {
          "serverCancelSupport": false,
          "requests": {
            "full": false,
            "range": false
          },
          "multilineTokenSupport": false,
          "dynamicRegistration": false,
          "overlappingTokenSupport": false,
          "tokenTypes": [
            "type",
            "class",
            "enum",
            "interface",
            "struct",
            "typeParameter",
            "parameter",
            "variable",
            "property",
            "enumMember",
            "event",
            "function",
            "method",
            "macro",
            "keyword",
            "modifier",
            "comment",
            "string",
            "number",
            "regexp",
            "operator"
          ],
          "tokenModifiers": [],
          "formats": [
            "relative"
          ]
        },
        "signatureHelp": {
          "dynamicRegistration": false
        },
        "definition": {
          "dynamicRegistration": false,
          "linkSupport": true
        }
      }
    },
    "rootPath": "/Users/lieryan/Projects/rope/pylsp-rope",
    "clientInfo": {
      "name": "vim-lsp"
    },
    "processId": 27105,
    "trace": "off"
  }
}

Server Capabilities

{
  "id": 1,
  "jsonrpc": "2.0",
  "result": {
    "capabilities": {
      "executeCommandProvider": {
        "commands": [
          "pylsp_rope.quickfix.generate",
          "pylsp_rope.refactor.introduce_parameter",
          "pylsp_rope.refactor.extract.method",
          "pylsp_rope.refactor.extract.variable",
          "pylsp_rope.refactor.inline",
          "pylsp_rope.refactor.local_to_field",
          "pylsp_rope.refactor.method_to_method_object",
          "pylsp_rope.refactor.use_function",
          "pylsp_rope.source.organize_import"
        ]
      },
      "documentHighlightProvider": true,
      "hoverProvider": true,
      "referencesProvider": true,
      "notebookDocumentSync": {
        "notebookSelector": [
          {
            "cells": [
              {
                "language": "python"
              }
            ]
          }
        ]
      },
      "signatureHelpProvider": {
        "triggerCharacters": [
          "(",
          ",",
          "="
        ]
      },
      "foldingRangeProvider": true,
      "codeActionProvider": true,
      "textDocumentSync": {
        "save": {
          "includeText": true
        },
        "change": 2,
        "openClose": true
      },
      "codeLensProvider": {
        "resolveProvider": false
      },
      "workspace": {
        "workspaceFolders": {
          "changeNotifications": true,
          "supported": true
        }
      },
      "definitionProvider": true,
      "documentRangeFormattingProvider": true,
      "documentFormattingProvider": true,
      "documentSymbolProvider": true,
      "experimental": {},
      "renameProvider": true,
      "completionProvider": {
        "resolveProvider": true,
        "triggerCharacters": [
          "."
        ]
      }
    },
    "serverInfo": {
      "version": "1.10.0",
      "name": "pylsp"
    }
  }
}

Server Configuration

{
  "method": "workspace/didChangeConfiguration",
  "params": {
    "settings": {
      "pylsp": {
        "plugins": {
          "pylsp_rope": {
            "rename": true,
            "enabled": true
          },
          "rope_rename": {
            "enabled": false
          }
        }
      }
    }
  }
}

Additional info

The logs are too long for Github, so I am attaching them as Gist.

• Full logs

[lieryan]

I tried to replicate the issue on coc.nvim, ale, and neovim native lsp, and they all behaved correctly in this situation.