Gathering environment information fails consistently

Hey folks! I'm just trying out SCIP for the first time. I wanted to try running it on my python repo and used the command scip-python index . and it is consistenly failing. Running pip3 show -f on the CLI works fine.

(13:48:15) Evaluating python environment dependencies
(13:48:17)   Gathering environment information from `pip`


Experienced Fatal Error While Indexing:
Please create an issue at github.com/sourcegraph/scip-python: <ref *1> Error: spawnSync /bin/sh ENOBUFS
    at Object.spawnSync (node:internal/child_process:1124:20)
    at spawnSync (node:child_process:876:24)
    at Object.execSync (node:child_process:957:15)
    at /opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:104453
    at t.withStatus (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:77273)
    at t.default (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:104241)
    at M.index (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:26081)
    at f (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:38106)
    at I.<anonymous> (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:3948)
    at I._actionHandler (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/vendor.js:2:1059335) {
  errno: -55,
  code: 'ENOBUFS',
  syscall: 'spawnSync /bin/sh',
  path: '/bin/sh',
  spawnargs: [
    '-c',
    'pip3 show -f aiohttp aiosignal aiostream annotated-types anthropic anyio asgiref attrs beautifulsoup4 black bleach blinker boltons boto3 boto3-stubs botocore botocore-stubs bracex certifi cffi cfgv charset-normalizer click click-option-group cloudpickle colorama coverage cryptography dataclasses-json defusedxml Deprecated distlib distro dnspython docker exa-py face fastapi fastjsonschema filelock flake8 Flask frozenlist fsspec gitdb GitPython glom grpclib h11 h2 hpack httpcore httpx huggingface-hub hyperframe identify idna importlib-metadata iniconfig isort itsdangerous Jinja2 jira jmespath jsonpatch jsonpointer jsonschema jsonschema-specifications jupyter_client jupyter_core jupyterlab_pygments jwt langchain langchain-community langchain-core langchain-openai langchain-pinecone langsmith loguru markdown-it-py marko MarkupSafe marshmallow mccabe mdurl mistune modal multidict mypy mypy-extensions nbclient nbconvert nbformat networkx networkx-stubs nodeenv numpy oauthlib openai packaging pandas pandas-stubs pandocfilters pathspec peewee pillow pinecone-client pip platformdirs pluggy pre-commit protobuf pycodestyle pycparser pydantic pydantic_core pyflakes PyGithub Pygments PyJWT PyNaCl pytest pytest-json-report pytest-metadata pytest-mock python-dateutil python-dotenv pytz PyYAML pyzmq redis referencing regex requests requests-oauthlib requests-toolbelt rich rpds-py ruamel.yaml ruamel.yaml.clib ruff s3transfer semgrep setuptools sigtools six slack-bolt slack_sdk smmap sniffio soupsieve SQLAlchemy starlette synchronicity tblib tenacity tiktoken tinycss2 tokenizers toml tomli tornado tqdm traitlets tree-sitter tree-sitter-languages typer types-awscrt types-certifi types-pytz types-PyYAML types-requests types-s3transfer types-setuptools types-toml types-tree-sitter types-tree-sitter-languages types-urllib3 typing_extensions typing-inspect tzdata unidiff urllib3 uvicorn virtualenv watchfiles watchtower wcmatch webencodings Werkzeug wrapt yarl zipp'
  ],
  error: [Circular *1],
  status: null,
  signal: 'SIGTERM',
  output: [
    null,
    <Buffer 4e 61 6d 65 3a 20 61 69 6f 68 74 74 70 0a 56 65 72 73 69 6f 6e 3a 20 33 2e 39 2e 31 0a 53 75 6d 6d 61 72 79 3a 20 41 73 79 6e 63 20 68 74 74 70 20 63 ... 1048618 more bytes>,
    <Buffer >
  ],
  pid: 74461,
  stdout: <Buffer 4e 61 6d 65 3a 20 61 69 6f 68 74 74 70 0a 56 65 72 73 69 6f 6e 3a 20 33 2e 39 2e 31 0a 53 75 6d 6d 61 72 79 3a 20 41 73 79 6e 63 20 68 74 74 70 20 63 ... 1048618 more bytes>,
  stderr: <Buffer >
}

Looks like it's because pip's output is too big to put in the buffer. A proper fix would involve rewriting this with child_process.spawn() (https://stackoverflow.com/questions/63796633/spawnsync-bin-sh-enobufs) but a workaround is to just give a larger buffer to exec:

diff --git a/packages/pyright-scip/src/virtualenv/environment.ts b/packages/pyright-scip/src/virtualenv/environment.ts
index 802ba9f8d..1abd0178b 100644
--- a/packages/pyright-scip/src/virtualenv/environment.ts
+++ b/packages/pyright-scip/src/virtualenv/environment.ts
@@ -29,13 +29,13 @@ let getPipCommand = () => {
 };
 
 function pipList(): PipInformation[] {
-    return JSON.parse(child_process.execSync(`${getPipCommand()} list --format=json`).toString()) as PipInformation[];
+    return JSON.parse(child_process.execSync(`${getPipCommand()} list --format=json`, { maxBuffer: 1024*1024*5 }).toString()) as PipInformation[];
 }
 
 function pipBulkShow(names: string[]): string[] {
     // TODO: This probably breaks with enough names. Should batch them into 512 or whatever the max for bash would be
     return child_process
-        .execSync(`${getPipCommand()} show -f ${names.join(' ')}`)
+        .execSync(`${getPipCommand()} show -f ${names.join(' ')}`, { maxBuffer: 1024*1024*5 })
         .toString()
         .split('\n---');
 }

I'm running into this as well. Any workarounds? I'm wiping my venv but let me know if there's a better way.

I'm facing the same issue , any updates on this?

Does this mean Sourcegraph is not really using this version of scip-python themself anymore? Surely they need to be able to index projects with large number of dependencies. The underlying pyright version is also getting quite old.

@olejorgenb

@olejorgenb we are using scip-python in production. Our team at has had limited capacity for the better part of the last ~2 years, maintaining several different indexers, as well as a bunch of code in the Sourcegraph backend, so we've been time constrained in what issues we can look at. This has meant that we have to had to mainly prioritize issues that customers and prospects are hitting.

I admit that we could've done a better job in triaging things here, as fixing this issue wasn't a big deal. I've landed a fix for this now in #193.

Navigation Menu

Repository navigation

Gathering environment information fails consistently #151

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Gathering environment information fails consistently #151

Description

Activity

zygi commented on Apr 24, 2024

drob commented on May 29, 2024

RheagalFire commented on Jul 29, 2024

olejorgenb commented on Jun 22, 2025

varungandhi-src commented on Sep 1, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions