![]() If it's Python 2.6.X, it's probably a good idea to use a recent build of Python 2.7 If it's Python 2.7.X, then you'll need to choose to use the system python or not. Verify your version of Python: python - version I've implemented it before and found THE JUPYTER DOCUMENTATION explains setting it up for encryption (HTTPS) and authentication to be pretty good. I'm leaving out Jupyter server mode security, which could be the topic of a future blog, potentially. Do all this using Jupyter in server mode that I access from my own laptop.Have access to modules like numpy, scipy, pandas and others.Run python code in YARN distributed mode.Run PySpark successfully from either a cluster node or an edge node. ![]() Hence my having so much trouble getting everything working to my satisfaction. It seems like it changed quite a bit since the earlier versions and so most of the information I found in blogs were pretty outdated. In fact, I've tested this to work with MapR 5.0 with MEP 1.1.2 (Spark 1.6.1) for a customer. It should work equally well for earlier releases of MapR 5.0 and 5.1. This article targets the latest releases of MapR 5.2.1 and the MEP 3.0 version of Spark 2.1.0. Having gone through the process myself, I've documented my steps and will share my knowledge, hoping it will save some time and frustration for some of you. I'll guess that many people reading this have spent time wrestling with a configuration to get Python and Spark to play nicely.
0 Comments
Leave a Reply. |