[Feature Request] Improve compatibility between PromptFlow and Python modules with relative imports

[pamelafox]

I am trying to use the promptflow-evals SDK in a project where I am using relative imports, which works fine because of how I call the modules (with python -m modulename).

However, PromptFlow tries to import my file for some reason, and then errors:

  File "/Users/pamelafox/ai-rag-chat-evaluator/scripts/evaluate.py", line 142, in run_evaluation
    results = evaluate(
              ^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/evals/evaluate/_evaluate.py", line 252, in evaluate
    input_data_df, target_generated_columns, target_run = _apply_target_to_data(target, data, pf_client,
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/evals/evaluate/_evaluate.py", line 128, in _apply_target_to_data
    run = pf_client.run(
          ^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_pf_client.py", line 301, in run
    return self._run(
           ^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_pf_client.py", line 226, in _run
    return self.runs.create_or_update(run=run, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_telemetry/activity.py", line 265, in wrapper
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/operations/_run_operations.py", line 134, in create_or_update
    created_run = RunSubmitter(client=self._client).submit(run=run, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/run_submitter.py", line 42, in submit
    self._run_bulk(run=run, stream=stream, **kwargs)
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/run_submitter.py", line 111, in _run_bulk
    with flow_overwrite_context(flow_obj, tuning_node, variant, connections=run.connections) as flow:
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/utils.py", line 264, in flow_overwrite_context
    override_flow_yaml(
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/utils.py", line 219, in override_flow_yaml
    update_signatures(code=flow_dir_path, data=flow_dag)
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_utilities/signature_utils.py", line 148, in update_signatures
    signatures, _, _ = infer_signature_for_flex_flow(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_utilities/signature_utils.py", line 72, in infer_signature_for_flex_flow
    flow_meta = inspector_proxy.get_entry_meta(entry=entry, working_dir=code)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_proxy/_python_inspector_proxy.py", line 47, in get_entry_meta
    return _generate_flow_meta(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_core/entry_meta_generator.py", line 84, in _generate_flow_meta
    raise GenerateFlowMetaJsonError(error_message)
promptflow.core._errors.GenerateFlowMetaJsonError: Generate meta failed, detail error:
["Failed to load python module from file '/Users/pamelafox/ai-rag-chat-evaluator/scripts/evaluate.py': (ImportError) attempted relative import with no known parent package"]

So now I have to restructure my project to avoid relative imports. Please loosen this constraint.

[D-W-]

Hi @pamelafox , in our concept, flow should be self-contained. Which makes it easier to build flow snapshot and share from local to cloud. When flow has relative external imports, we could not build snapshot for it. But we can improve the error message to make it more friendly to customer.