Exam Certified Data Engineer Professional topic 1 question 226 discussion

Correct: B - To use common Python testing frameworks (like pytest or unittest) effectively, code should be structured as standard Python modules (.py files) rather than notebooks. Databricks "Files in Repos" (now often called Databricks Git Folders or Workspace Files) allows you to create these .py files alongside your notebooks. You can then import the functions into your notebooks for execution and simultaneously import them into test files to run unit tests, enabling standard CI/CD workflows.

suretly C

B - to test functions we need to import them in our unit tests. This means that storing functions in notebooks is not a good idea. You store them separately - as "files" - and import in notebooks the same way you import them in unit tests.

Databricks Repos is the recommended way to organize and manage code, including functions and unit tests, in a scalable and maintainable way. By defining your functions and unit tests in Files in Repos, you can: Modularize Your Code: Functions can be organized into separate Python files or modules, making them reusable and easier to test. Use Standard Testing Frameworks: Frameworks like pytest or unittest can be used to write and execute unit tests against these functions. Version Control Integration: Files in Repos can be version-controlled using Git, ensuring traceability and collaboration. Production Data Testing: With proper safeguards, you can design unit tests to test production-like data while maintaining modularity and separation from production pipelines.

It is true that or Python and R notebooks, Databricks recommends storing functions and their unit tests outside of notebooks so given the question is in Python environment some could argue B being the correct solution. Nevertheless that is only an "advice" and Databricks more generally states that in general, it is a best practice to not run unit tests against functions that work with data in production. This is especially important for functions that add, remove, or otherwise change data. To protect your production data from being compromised by your unit tests in unexpected ways, you should run unit tests against non-production data. So I would got for option C. Alternatively I would have gone for B. https://docs.databricks.com/en/notebooks/testing.html?utm_source=chatgpt.com#write-unit-tests