{"author":"Oracle Cloud Infrastructure Data Science","author_email":null,"bugtrack_url":null,"classifiers":["Development Status :: 5 - Production/Stable","Intended Audience :: Developers","License :: OSI Approved :: Universal Permissive License (UPL)","Operating System :: OS Independent","Programming Language :: Python :: 3.10","Programming Language :: Python :: 3.11","Programming Language :: Python :: 3.12","Programming Language :: Python :: 3.6","Programming Language :: Python :: 3.7","Programming Language :: Python :: 3.8","Programming Language :: Python :: 3.9"],"description":"\n# Oracle Cloud Infrastructure Object Storage fsspec Implementation\n\n\n[![PyPI](https://img.shields.io/pypi/v/ocifs.svg?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/ocifs/) [![Python](https://img.shields.io/pypi/pyversions/ocifs.svg?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/ocifs/)\n\n\n\nThe [Oracle Cloud Infrastructure Object Storage](https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/objectstorageoverview.htm) service is an internet-scale, high-performance storage platform that offers reliable and cost-efficient data durability. With Object Storage, you can safely and securely store or retrieve data directly from the internet or from within the cloud platform.\n\n`ocifs` is part of the `fsspec` [intake/filesystem_spec ecosystem](https://github.com/intake/filesystem_spec)\n\n> a template or specification for a file-system interface, that specific implementations should follow, so that applications making use of them can rely on a common interface and not have to worry about the specific internal implementation decisions with any given backend.\n\n`ocifs` joins the list of file systems supported with this package.\n\nThe `intake/filesystem_spec` project is used by [Pandas](https://pandas.pydata.org/), [Dask](https://dask.org/) and other data libraries in python, this package adds Oracle OCI Object Storage capabilties to these libraries.\n\n## OCIFS file system style operations Example:\n```python\nfrom ocifs import OCIFileSystem\n\nfs = OCIFilesystem(\"~/.oci/config\")\n# 1.Create empty file or truncate in OCI objectstorage bucket\n fs.touch(\"oci://@//hello.txt\", truncate=True, data=b\"Writing to Object Storage!\")\n # 2.Fetch(potentially multiple paths' contents\n fs.cat(\"oci://@//hello.txt\")\n # 3.Get metadata about a file from a head or list call\n fs.info(\"oci://@//hello.txt\")\n # 4.Get directory listing page\n fs.ls(\"oci://@//\", detail=True)\n # 5.Is this entry directory-like?\n fs.isdir(\"oci://@\")\n # 6.Is this entry file-like?\n fs.isfile(\"oci://@//hello.txt\")\n # 7.If there is a file at the given path (including broken links)\n fs.lexists(\"oci://@//hello.txt\")\n # 8.List of files for the given path\n fs.listdir(\"oci://@/\", detail=True)\n # 9.Get the first ``size`` bytes from file\n fs.head(\"oci://@//hello.txt\", size=1024)\n # 10.Get the last ``size`` bytes from file\n fs.tail(\"oci://@//hello.txt\", size=1024)\n # 11.Hash of file properties, to tell if it has changed\n fs.ukey(\"oci://@//hello.txt\")\n # 12.Size in bytes of file\n fs.size(\"oci://@//hello.txt\")\n # 13.Size in bytes of each file in a list of paths\n paths = [\"oci://@//hello.txt\"]\n fs.sizes(paths)\n # 14.Normalise OCI path string into bucket and key.\n fs.split_path(\"oci://@//hello.txt\")\n # 15.Delete a file from the bucket\n fs.rm(\"oci://@//hello.txt\")\n # 16.Get the contents of the file as a byte\n fs.read_bytes(\"oci://@//hello.txt\", start=0, end=13)\n # 17.Get the contents of the file as a string\n fs.read_text(\"oci://@//hello.txt\", encoding=None, errors=None, newline=None)\n # 18.Get the contents of the file as a byte\n fs.read_block(\"oci://@//hello.txt\", 0, 13)\n # 19.Open a file for writing/flushing into file in OCI objectstorage bucket\n # Ocifs sets the best-guessed content-type for hello.txt i.e \"text/plain\"\n with fs.open(\"oci://@//hello.txt\", 'w', autocommit=True) as f:\n f.write(\"Writing data to buffer, before manually flushing and closing.\") # data is flushed and file closed\n f.flush()\n# Ocifs uses the specified content-type passed in the open while writing to OCI objectstorage bucket\n with fs.open(\"oci://@//hello.txt\", 'w',content_type='text/plain') as f:\n f.write(\"Writing data to buffer, before manually flushing and closing.\") # data is flushed and file closed\n f.flush()\n # 20.Open a file for reading a file from OCI objectstorage bucket\n with fs.open(\"oci://@//hello.txt\") as f:\n print(f.read())\n # 21.Space used by files and optionally directories within a path\n fs.du(\"oci://@//hello10.csv\")\n # 22.Find files by glob-matching.\n fs.glob(\"oci://@//*.txt\")\n # 23.Renames an object in a particular bucket in tenancy namespace on OCI\n fs.rename(\"oci://@//hello.txt\", \"oci://@//hello2.txt\")\n # 24.Delete multiple files from the same bucket\n pathlist = [\"oci://@//hello2.txt\"]\n fs.bulk_delete(pathlist)\n\n```\n\n\n\n### Or Use With Pandas\n\n```python\nimport pandas as pd\nimport ocifs\n\ndf = pd.read_csv(\n \"oci://my_bucket@my_namespace/my_object.csv\",\n storage_options={\"config\": \"~/.oci/config\"},\n)\n```\n\n### Or Use With PyArrow\n\n```python\nimport pandas as pd\nimport ocifs\n\ndf = pd.read_csv(\n \"oci://my_bucket@my_namespace/my_object.csv\",storage_options={\"config\": \"~/.oci/config\"})\n```\n\n### Or Use With ADSDataset\n\n```python\nimport ads\nimport pandas as pd\nfrom ads.common.auth import default_signer\nfrom ads.dataset.dataset import ADSDataset\n\n\n ads.set_auth(auth=\"api_key\", oci_config_location=\"~/.oci/config\", profile=\"\")\n ds = ADSDataset(\n df=pd.read_csv(f\"oci://my_bucket@my_namespace/my_object.csv\", storage_options=default_signer()),\n type_discovery=False\n )\n print(ds.df)\n```\n\n\n## Getting Started\n```bash\npython3 -m pip install ocifs\n```\n\n## Software Prerequisites\nPython >= 3.6\n\n## Environment Variables for Authentication:\n```bash\nexport OCIFS_IAM_TYPE=api_key\nexport OCIFS_CONFIG_LOCATION=~/.oci/config\nexport OCIFS_CONFIG_PROFILE=DEFAULT\n```\n\nNote, if you are operating on OCI with an alternative valid signer, such as resource principal, instead set the following:\n```bash\nexport OCIFS_IAM_TYPE=resource_principal\n```\n\n## Environment Variables for enabling Logging:\nTo quickly see all messages, you can set the environment variable OCIFS_LOGGING_LEVEL=DEBUG.\n```bash\nexport OCIFS_LOGGING_LEVEL=DEBUG\n```\n\n## Documentation\n* [![PyPI](https://img.shields.io/pypi/v/ocifs.svg?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/ocifs/) [![Python](https://img.shields.io/pypi/pyversions/ocifs.svg?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/ocifs/)\n* [ocifs Documentation](https://ocifs.readthedocs.io/en/latest/index.html)\n* [ocifs GitHub](https://github.com/oracle/ocifs)\n\n## Support\n[The built-in filesystems in `fsspec`](https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations) are maintained by the `intake` project team, where as `ocifs` is an external implementation (similar to `s3fs`, `gcsfs`, `adl/abfs`, and so on), which is maintained by Oracle.\n\n## Contributing\nThis project welcomes contributions from the community. Before submitting a pull request, please [review our contribution guide](./CONTRIBUTING.md)\n\n## Security\nPlease consult the [security guide](./SECURITY.md) for our responsible security vulnerability disclosure process\n\n## License\nCopyright (c) 2021, 2023 Oracle and/or its affiliates.\n\nReleased under the Universal Permissive License v1.0 as shown at\n.\n\n","description_content_type":"text/markdown","docs_url":null,"download_url":null,"downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":null,"keywords":"Oracle Cloud Infrastructure, OCI, Object Storage","license":null,"license_expression":null,"license_files":null,"maintainer":null,"maintainer_email":"Allen Hosler ","name":"ocifs","package_url":"https://pypi.org/project/ocifs/","platform":null,"project_url":"https://pypi.org/project/ocifs/","project_urls":{"Documentation":"https://ocifs.readthedocs.io/en/latest/index.html","Github":"https://github.com/oracle/ocifs"},"provides_extra":null,"release_url":"https://pypi.org/project/ocifs/1.3.2/","requires_dist":["fsspec>=0.8.7","oci>=2.43.1","requests"],"requires_python":">=3.6","summary":"Convenient filesystem interface over Oracle Cloud's Object Storage","version":"1.3.2","yanked":false,"yanked_reason":null}

Dependencies of ocifs (3).
Dependency	Optional
fsspec	false
oci	false
requests	false

Package search

ocifs

project_urls

Dependencies

All package metadata

Releases