The Art of Visualizing High Dimensional Data
Finally published! This article discusses enriched visualizations, with a focus on animated gifs and videos built in Python. For instance, the comet video can feature several dimensions that are difficult to show in a static picture: the comet locations at any given time, the relative velocity of each comet, the change in velocity (acceleration), the change in comet size when approaching the sun, the comet interactions (the apparent collisions), and more. It can easily display 17 dimensions, as discussed in the paper.
The PDF document (6 pages + code + illustrations, 11MB) focuses on four applications: prediction intervals in any dimension, supervised classification, convergence of algorithms such as gradient descent when dealing with chaotic functions, and spatial time series (the comet illustration). All visualizations use the RGB color model, and one uses RGBA for special and particularly useful effects, by playing with the transparency level. In essence it allows you to perform supervised classification using image techniques only, after mapping your dataset onto an image.
Image compression and anti-aliasing techniques are included in the Python code. They require only a simple call to a library function. The code is also on GitHub, and the videos on YouTube. The document also presents surprising data in number theory and experimental math. It leads to interesting machine learning problems: boundary / holes detection, and convergence acceleration for chaotic iterations.
Read the full article and access the free PDF, here .
Lead Machine Learning Engineer | Ex-Accenture | Innovator - 7 Patents | Machine Learning Mentor
2 年Thanks for sharing. Will read through the PDF, install the libraries and keep you posted. I am used to tSNE Curious to know, how what is the USP of the approach vs normal tSNE? Vincent Granville
Product Lead: Cisco Cloud Security: Public Sector Market Expansion
2 年Congrats Vincent! Look forward to a close read...
Chairman, President & CEO at Synergism, Inc. and Owner, Synergism, Inc.
2 年Shouldn't the comets in the simulation have scattering after the collisions?
Chairman, President & CEO at Synergism, Inc. and Owner, Synergism, Inc.
2 年I have accessed the pdf and read the article However, It is not playing out to experience the results. Do I need to install additional software?
I create. I build.
2 年We just did planar slices. Partial derivatives. Only takes you so far, however and the data has/had to be set up right to data viz in 3 dimensions. The most we could handle was 7 before things got hairy. We could have added in 3 more for color but there's a point where the human mind can only keep track of so much at once. We did this on an IBM RISC-6000 with Data Xplorer.