On Wednesday, March 6th, the SecondMuse team brought together startups from its two NYC-based manufacturing programs – Futureworks Incubator and M-Corps – for another session in the M-Corps workshop series. Hosted by M-Corps partner New Lab in the Brooklyn Navy Yard, this workshop focused on Quality Assurance (QA).

Founders and teams in attendance heard from a panel of three industry experts– Bob Kowalski, Senior Director of Mechanical Engineering at Intelligent Product Solutions, a cross functional design services and manufacturing company; Diana Pincus, VP of Operations at LittleBits, an award-winning modular electronics platform; and Jim Griszbacher, former VP of Engineering at Teralytics– as well as guest speaker Suma Reddy, COO and Co-Founder of M-Corp’s cohort company Farmshelf. Suma’s experience building and scaling their indoor farming platform for high-end restaurants and hotels served as the perfect case study for lessons learned (so far) about QA management.

M-Corps, funded by NYSERDA and sponsored by Dragon Innovation, launched in September 2018 to help cleantech startups get to market faster through easing the pain of manufacturing. We’re currently supporting a cohort of 11 companies that range from kits that make commercial-grade vehicles electric to consumer solar products, smart ice-melt control systems to naturally occurring soil remediation products. The M-Corps workshop series is based on common needs from startups in SecondMuse programs, and each workshop is open to founders across the manufacturing hardware ecosystem.

Check out posts about our prior workshops – Supply Chain Strategy & Manufacturing 101: Talking to Manufacturers. View the presentation from our QA workshop and explore key takeaways from the session below.

Key Takeaways

  • Start early with quality assurance. You’ll pay for it later if you don’t.
  • Track your parts in a consistent manner, such as using unique serial numbers.
  • Document your process and assembly flow. Make sure it can be easily shared with a manufacturer.
  • Self-test your product as much as possible.

Quality Assurance

Quality Assurance doesn’t end when a product leaves the assembly line; it matters throughout the entire  life of a product. Everything starts with a great Quality Assurance plan. While teams shouldn’t expect to catch every issue from the start, they do need a flexible process in place that they can adapt to quality issues and quickly incorporate more tests to minimize those issues going forward.

Diana Pincus from LittleBits recommends thinking through how your customers will use your product to find failure points. Run mock exercises to simulate how the end user will use (and abuse) your product. For example, when Diana worked at Makerbot, they ran new printers for thousands of hours to test them and identify any issues that might arise from sustained use. When not running test prints, the team would simulate other real-life use cases and failure points, such as performing drop tests to see how the printers held up after falling off a table.

Here’s an example of a test plan for a new hardware product (also known as an Engineering Validation Test (EVT) or Design Validation Test (DVT):

  1. Functional test first – Test your electronics using test fixtures
  2. Assembly – screws and cables
  3. Sub assembly Test 1 – Build your product. If you have a problem during the build, you can reclaim and replace some materials or components in the design.
  4. Sub assembly Test 2 – Glue your product shut. With this test, you may realize that you create a bunch of scrap because you can’t re-use that component.
  5. Complete sub-assembly – Does it all come together? Does it “work” like it’s supposed to? If not, what are the changes you should make to the design or engineering to correct those issues?

All three facilitators stressed the importance of starting testing early, not only to catch errors, but to avoid scrap. Scrap will inevitably accumulate as a product is tested and refined, but minimizing how much your process produces can reduce the complexity of your operations, the waste you produce, and your cost of production.  

Pro Tip:

When it comes to working with contract manufacturers (CMs), don’t hesitate to give them a QA plan if you already have one. CMs often have the capability to implement a QA process more cost effectively than a startup can on their own. Ultimately, your CM wants you to succeed, because they only get paid if you receive your product(s). Don’t be afraid to ask your CM to share their QA plan!

QA Tools

To implement an airtight QA process, consider using the following tools:

Design Reviews

Bring in outside reviewers to review your product frequently. Seek out expert feedback on your design for manufacturing and design for assembly.

Failure Mode and Effects Analysis (FMEA)

Test and analyze your product’s failure points. This is where those drop tests Diana Pincus performed at Makerbot come in. On April 3, we are hosting a Risk Management & Assembly Workshop on this very topic.

Statistical Tolerance Analysis

Make drawings with tolerances linked to functional requirements and run statistical models to ensure fit. This will save you a lot of headaches in the future when you try to fit everything together. If you’re now sure what tolerances your product can or should handle, research similar products on the market and see what they can withstand.

First Article of Inspection

Check every dimension to confirm what is to spec. Be practical about this; if it is something that is going to affect functionality or delay production, you may want to leave it out.

Engineering  Validation Test

If you have a set of requirements, make sure everything passes. Run drop tests, test how your product holds up under variable weights, etc. Note that not all tests are physical!

Reliability Testing (Design Validation Testing)

Validate how long your product will last under real-life conditions. Try to find the weak links; this will help you prioritize what to fix in your design.

Documentation!

Clearly define requirements physically and cosmetically and track your testing, the results, and changes made to the design or components. This will help keep your BOM current.

Value Stream Map

Map out where everything comes from. You’ll have a clearer view of the different potential risks, the time involved, or other key pieces and people that are required.

Testing Stations and Checkpoints

Work with your manufacturer to set up testing points along your production line. Your CM can provide guidance for some of these (ie: FMEA, reliability and engineering validation testing), but as an entrepreneur or engineer at a young startup, manage design reviews, first article of inspection, and documentation of your process to ensure you and your team are intimately familiar with problems that need to be addressed and the process by which they will be.

The Root of It All

  • It won’t always be clear what is at the core of a QA problem, so working with your team and CM to drill down into the root cause is essential. Our facilitators discussed a handful of strategies to help you get there:
  • 5 Whys. This is an actual root cause analysis technique used by product teams and manufacturers. With 5 Whys, you and your team will interrogate the problem, asking simple questions about why various failure points are occuring. The point is to keep stepping back from the problem–by the time you get to the 5th Why, you’ll likely understand what needs to be fixed.
  • Fishbone Diagram. Also called an Ishikawa diagram, Fishbone Diagrams show the cause of an event. Develop your diagram as a group exercise; write out every possible cause for the problem and then prioritize what to focus on and understand where to put time and resources.
  • Pareto Chart. Pareto Charts help you understand where most of your problems are coming from, so you can hone in on your top priorities. Your team lists out all of the problems encountered and the number of occurrences of that problem. You’ll find that 80% of problems come from 20% of causes.
  • Request an 8D (8 Disciplines) and corrective action report. Originally developed by Ford, an 8D report walks you through a manufacturer’s process of investigation. CMs should always be willing to share their process for sorting problems and the steps they would take to resolve them.

One important point to remember: when it comes to CMs and quality assurance, you have to ask for it, otherwise they may not plan for it. This may be a challenge for startups with low production volumes that may not have a lot of leverage or cash to make requests with. In these cases, it’s really about building your relationship with your CM and helping them see your potential so they get excited about your product. Treat this stage of your process as a collaboration with your CM; you’re building this product and solving QA issues together. Don’t blame them for problems, but be firm about getting ahead of them when they occur.

Lastly, build your quality assurance strategy around your customer, not your product, because you always have to balance quality and revenue. Your customers, and how they use the product, should be top of mind as you strategize  what problems to tackle first. Don’t worry as much about the edge cases.

Case Study: Suma Reddy from Farmshelf

Farmshelf’s product has a lot of parts and subassemblies, some of which may be exposed to light, water, and soil. Sheet metal, glass, and injection molded parts are used to house cameras, LEDs, custom PCBAs; and LED boards make up the brains and interior of the product. That’s a lot of complexity and potential failure points. To know their manufacturing and assembly process inside and out, the team chose to build 100 units in-house, meaning it is essential for them to develop their own QA/QC processes.

  • This is how they made there In-House QA Plan:
  • Implementation of a serial number-tracking system for logging and tracking every component of the Farmshelf. This allowed them to drill down at the component level if there was an issue.
  • In-house production has proven to be a heavy lift, which prompted them to accelerate their conversations with an offshore CM.
  • Avoid “unbuilding” if you don’t have to. Make adjustments for quality by adding or simplifying parts of the product is one thing, but unraveling subassemblies after they have been built because an error was found late in the game is painful (and expensive). Work with your CM to ensure this doesn’t happen.
  • Understand your production line, people, process, and how to setup your line to minimize damage. Think about all of this early in your process, because you will have to eventually. Not knowing where most of your problems are coming from can be quite costly over time.
  • Most important: document, document, document. Suma and her team built Assembly Flow Diagrams and SOPs to engage with CMs. Start this early because you’ll need your documentation to scale, especially overseas.

Found this post useful? Come to our upcoming workshops!

Tags

Risk Assessment