--- description: Review of Pachyderm Software: system overview, features, price and cost information. Get free demos and compare to similar programs. image: https://gdm-localsites-assets-gfprod.imgix.net/images/software_advice/og_logo-55146305bbe7b450bea05c18e9be9c9a.png title: Pachyderm | Reviews, Pricing & Demos - SoftwareAdvice GB --- Breadcrumb: [Home](/) > [Deep Learning Software](/directory/4392/deep-learning/software) > [Pachyderm](/software/424256/pachyderm) # Pachyderm Canonical: https://www.softwareadvice.co.uk/software/424256/pachyderm Page: 1 / 2\ Next: [Next page](https://www.softwareadvice.co.uk/software/424256/pachyderm?page=2) > Company: Pachyderm is the leader in data versioning and pipelines for MLOps. We provide the data foundation that allows data science teams to automate and scale their machine learning lifecycle while guaranteeing reproducibility. With over $40 million in three rounds of funding from leading investors like Benchmark, Microsoft M12, Y Combinator, and others, Pachyderm, Inc. offers a commercial Pachyderm Enterprise Edition and an open source Pachyderm Community Edition. Pachyderm helps customers get their ML and AI projects to market faster, lower data processing and storage costs, and supports strict data governance requirements.. Products: Pachyderm is for data science teams who want to operationalize the data tasks in their ML lifecycle to iterate on data more quickly and reliably. Pachyderm is the leader in data versioning and pipelines for MLOps, and this data foundation allows data science teams to automate and scale their machine learning lifecycle while guaranteeing reproducibility. Unlike other data versioning and pipeline products Pachyderm provides data-driven automation, petabyte scalability and end-to-end reproducibility. Pachyderm Enterprise Edition: Pachyderm Enterprise Edition is our commercial offering designed for the largest projects in highly secure environments. Along with world-class support, your team also gets access to our full range of premium features including Pachyderm Console, authentication and access controls (RBAC), no scaling limits, JupyterHub integration, and centralized multiple cluster management. Pachyderm Community Edition: Pachyderm Community Edition is our open source version of Pachyderm. With Pachyderm Community Edition, you get the core Data Versioning and Pipeline features of Pachyderm that you can deploy locally or in the cloud of your choosing. If you need help, there’s an entire community of experts ready to offer their assistance. > > Verdict: Rated **4.0/5** by 7 users. Top-rated for **Likelihood to recommend**. ----- ## Quick Stats & Ratings | Metric | Rating | Detail | | **Overall** | **4.0/5** | 7 Reviews | | Ease of Use | 3.3/5 | Based on overall reviews | | Customer Support | 4.9/5 | Based on overall reviews | | Value for Money | 4.0/5 | Based on overall reviews | | Features | 4.6/5 | Based on overall reviews | | Recommendation percentage | 90% | (9/10 Likelihood to recommend) | ## About the vendor - **Company**: Hewlett Packard Enterprise ## Commercial Context - **Pricing model**: Usage Based (Free Trial) - **Pricing Details**: https://www.pachyderm.com/trial/ - **Target Audience**: 51–200, 201–500, 501–1,000, 1,001–5,000, 5,001–10,000, 10,000+ - **Deployment & Platforms**: Cloud, SaaS, Web-based, Mac (Desktop), Linux (Desktop), Linux (On-Premise) - **Supported Languages**: English - **Available Countries**: United States ## Features - API - Access Controls/Permissions - Activity Dashboard - Asynchronous Learning - Collaboration Tools - Compliance Management - Configurable Workflow - Data Capture and Transfer - Data Cleansing - Data Connectors - Data Extraction - Data Import/Export - Data Storage Management - Data Transformation - Data Visualisation - Deep Learning - For eCommerce - High Volume Processing - Image Analysis - ML Algorithm Library ## Support Options - Email/Help Desk - FAQs/Forum - Knowledge Base ## Category - [Deep Learning Software](https://www.softwareadvice.co.uk/directory/4392/deep-learning/software) ## Related Categories - [Deep Learning Software](https://www.softwareadvice.co.uk/directory/4392/deep-learning/software) - [Artificial Intelligence (AI) Software](https://www.softwareadvice.co.uk/directory/4360/artificial-intelligence/software) - [Machine Learning Software](https://www.softwareadvice.co.uk/directory/4358/machine-learning/software) - [Big Data Software](https://www.softwareadvice.co.uk/directory/3299/big-data/software) ## Alternatives 1. [Anaconda](https://www.softwareadvice.co.uk/software/306432/anaconda) — 4.6/5 (86 reviews) 2. [Google Cloud](https://www.softwareadvice.co.uk/software/172144/google-cloud-platform) — 4.7/5 (2262 reviews) 3. [Splunk Enterprise](https://www.softwareadvice.co.uk/software/234262/splunk-enterprise) — 4.6/5 (259 reviews) 4. [Zerve](https://www.softwareadvice.co.uk/software/522850/Zerve) — 5.0/5 (2 reviews) 5. [OpenText Analytics Cloud](https://www.softwareadvice.co.uk/software/415497/opentext-magellan) — 5.0/5 (1 reviews) ## Reviews ### "Rethinking Data in AI and ML" — 4.0/5 > **Clayton** | *11 November 2021* | Hospital & Health Care | Recommendation rating: 10.0/10 > > **Pros**: AI/ML production systems typically consist of multiple data processing steps organized as a DAG. Many automation frameworks manage these DAGs as tightly coupled steps ordered by \_code execution\_. What I like so much about Pachyderm is that it approaches DAG management as loosely coupled steps ordered by \_data dependencies\_. This alternative way of thinking has enabled me to design AI/ML architectures with data at the center, which has revolutionized the development and production workflows I've participated in. I can confidently store, process, and otherwise manage the data because Pachyderm provides a solid foundation for data provenance, data versioning, data storage patterns, and efficient incremental processing. Since AI/ML models are effectively a form of data, model versioning and management can be built as an extension of Pachyderm's data foundation. Furthermore, I really like that Pachyderm is powered by Kubernetes, because it passes on important architectural properties to Pachyderm, such as high scalability, robustness, efficiency, and portability (i.e. cloud agnosticism). I can containerize my pipelines, quickly test them locally through Docker Desktop or minikube, then scale them up to massive amounts of data in an on-prem or cloud cluster. If autoscaling is supported in a cloud cluster, I can especially reap the benefits of cost efficiency because I only pay for the compute resources I use. > > **Cons**: - In 1.X versions of Pachyderm, there are a few performance pain points, especially around handling very small files when uploading/downloading to/from a repo. These pain points have been significantly improved in Pachyderm 2.X. - Also in 1.X, debugging pipeline failures can sometimes be challenging without extra tools or integrating external logging services. Pachyderm 2.X improves upon this as well. - When Pachyderm processes data files in a pipeline, it groups the files into logical structures called datums for provenance and data efficiency reasons, and then it invokes the pipeline on each datum. This is necessary for scalability, but the downside is that each invocation of the pipeline incurs an overhead cost of just starting the processing code. The bright side is that there are several straightforward ways to engineer around the problem. It's also important to recognize that the impact of the problem is minimized by the benefits of incremental processing(i.e. only processing data that has changed on future pipeline runs). - This isn't necessarily a problem, but prospective buyers should be aware that although compute costs may go down due to incremental processing, storage costs may go up due to storing multiple versions of data. > > Like any tool, Pachyderm is no silver bullet for the entire AI/ML stack. However, from a data processing and management perspective, it has fulfilled every application requirement I've needed it for and continues to be a flexible tool in meeting additional requirements. For example, after having computed some results from a pipeline, I needed to serve these results to an existing application. Pachyderm made this simple by exposing the data through a built-in S3 REST API. Since the application was already compatible with S3, Pachyderm served as a drop-in replacement for an S3 bucket. For anyone that strives to design clean and straightforward AI/ML architectures, I can definitely recommend Pachyderm as a must for the foundational data component. ----- ### "Game changer for handling dynamic data" — 4.0/5 > **Cove** | *17 November 2021* | Research | Recommendation rating: 10.0/10 > > **Pros**: Perhaps the most important aspect we benefit from operationally is the awareness and automatic handling of data change. Generation of our data products involves multiple processing steps and several sources of data and metadata that enter the processing sequence at various points and may change at any time. Pachyderm automatically knows what has changed and triggers downstream (re)processing, removing the need for error-prone human management. > > **Cons**: In Pachyderm 1.X there was a relatively high amount of overhead associated with processing each datum. Our data typically consists of small but numerous datums, and we needed to artificially combine datums for performance. However, Pachyderm has been working with us on this issue and we expect to see big improvements in 2.0 and beyond. > > Pachyderm meets many previously unmet needs for our organization, including complete data provenance, automatic handling of data change, and modular/portable processing architecture, which facilitates the joint development of processing pipelines between software developers and scientists. Pachyderm engineers have been extremely responsive to our issues and development requests, and we plan to work well into the future with this software. ----- ### "Great in theory" — 3.0/5 > **Martin** | *26 October 2021* | Biotechnology | Recommendation rating: 6.0/10 > > **Pros**: Great concept, really fits what we would like to do. Re-computing only the pieces where the data has changed is super valuable. > > **Cons**: Working with it in practice is very hard. We would like to use Pachyderm also for research, developing research pipelines that can be executed easily on big amounts of data on the cluster. However, during research/development, pipelines naturally crash often. Translating something that works locally to something that works in pachyderm has several scenarios in which it can fail. Inspecting those types of errors is incredibly difficult, unless you invest a significant amount of time into setting up logging/monitoring manually. > > We achieved some of our goals with Pachyderm. However, we were really hoping to spend more time on solving the problems directly related with our goal. Instead, we spent a significant amount on time solving problems with Pachyderm and tailoring our problem to it. ----- ### "Pachyderm is a great data processing platform on cloud." — 4.0/5 > **Xubo** | *25 October 2021* | Biotechnology | Recommendation rating: 9.0/10 > > **Pros**: Data Driven Automation. It supports incremental data processing. Reproducibility. Perfectly match our tech stacks: K8s, S3. Community facing. > > **Cons**: We expect fully automated data replication/export to external storage system. The logging \& debugging support could be improved. > > We have used Pachyderm for more than a year. Overall experience is Good. We love the core technology and features provided by Pachyderm. We experienced frustrated issues, like the download speed, deployment, system stability. We get excellent support from the Pachyderm team all the time. ----- ### "Scalable machine learning without the mlops" — 5.0/5 > **Chris** | *29 October 2021* | Marketing & Advertising | Recommendation rating: 10.0/10 > > **Pros**: The ability to scale model builds in native python is something that has been missing in this space until now. Utilizing spark and/or dask comes with a large amount of overhead that can be avoided leveraging pachyderm. > > **Cons**: The learning curve is quite steep since there are some core concepts that are foundational to understand before using pachyderm. ----- Page: 1 / 2\ Next: [Next page](https://www.softwareadvice.co.uk/software/424256/pachyderm?page=2) ## Links - [View on SoftwareAdvice](https://www.softwareadvice.co.uk/software/424256/pachyderm) ## This page is available in the following languages | Locale | URL | | en | | | en-AU | | | en-GB | | | en-IE | | | en-NZ | | ----- ## Structured Data