Multi Image Information Extract and Graphing through multi prompt engineering using Claude 3.5

Objective

To Demonstrate Claude’s multimodal Information extract and synthesis capabilities

?

Process Steps

·????? We load Two images to Claude which have related but complimentary information

·????? We prompt Claude to analyze the multi image input and generate a single JSON format of the data from the image files.

·????? As a second step we prompt Claude to generate a Single Graph presenting the information in a combined way using the JSON file generated .

STEP 1 : Upload image ?and prompt to Claude.AI

Step 2a: Check Response from Claude

Step 2b: Check Response from Claude – Check JSON

Step 3: Second Prompt

Step 4a: Check response from Claude

Step 4b: Check Generated Graph (click preview)

Step 4b: Check Generated React JS Code (click code)

import { useState } from 'react';
import { LineChart, Line, XAxis, YAxis, Tooltip, CartesianGrid, ReferenceLine } from 'recharts';
import { Card, CardContent, CardHeader } from '@/components/ui/card';
?
const data = [
? { year: 2001, cost: 100000000 },
? { year: 2002, cost: 50000000 },
? { year: 2003, cost: 25000000, milestone: "Human Genome Project completed" },
? { year: 2004, cost: 15000000 },
? { year: 2005, cost: 10000000, milestone: "International HapMap Project" },
? { year: 2006, cost: 8000000 },
? { year: 2007, cost: 7000000 },
? { year: 2008, cost: 3000000, milestone: "1000 Genomes Project" },
? { year: 2009, cost: 100000 },
? { year: 2010, cost: 50000, milestone: "Neanderthal genome draft" },
? { year: 2011, cost: 10000 },
? { year: 2012, cost: 8000, milestone: "eMERGE & UK Biobank" },
? { year: 2013, cost: 6000, milestone: "GA4GH" },
? { year: 2014, cost: 5000, milestone: "H3Africa & TOPMed" },
? { year: 2015, cost: 4000, milestone: "All of Us biobank" },
? { year: 2016, cost: 2000, milestone: "GWAS Catalog" },
? { year: 2017, cost: 1500, milestone: "PAGE & IGNITE" },
? { year: 2018, cost: 1000 },
? { year: 2019, cost: 900, milestone: "Human Pangenome Reference" },
? { year: 2020, cost: 800, milestone: "T2T X chromosome" },
? { year: 2021, cost: 700 },
? { year: 2022, cost: 600 }
];
?
export default function GenomeTimeline() {
? const [tooltipContent, setTooltipContent] = useState(null);
?
? return (
??? <Card className="w-full p-4">
????? <CardHeader>
??????? <h2 className="text-2xl font-bold text-center">Genome Sequencing Cost & Milestones Timeline</h2>
????? </CardHeader>
????? <CardContent>
??????? <div className="w-full h-[600px]">
????????? <LineChart
??????????? width={1000}
??????????? height={500}
??????????? data={data}
??????????? margin={{ top: 20, right: 30, left: 80, bottom: 100 }}
????????? >
??????????? <CartesianGrid strokeDasharray="3 3" />
??????????? <XAxis
????????????? dataKey="year"
????????????? angle={-45}
????????????? textAnchor="end"
????????????? height={60}
??????????? />
??????????? <YAxis
????????????? scale="log"
????????????? domain={['auto', 'auto']}
????????????? tickFormatter={(value) => `$${value.toLocaleString()}`}
????????????? label={{ value: 'Cost (USD)', angle: -90, position: 'insideLeft' }}
??????????? />
??????????? <Tooltip
????????????? formatter={(value) => [`$${value.toLocaleString()}`, 'Cost']}
????????????? labelFormatter={(label) => `Year: ${label}`}
??????????? />
??????????? <Line
????????????? type="monotone"
????????????? dataKey="cost"
????????????? stroke="#2563eb"
????????????? strokeWidth={2}
????????????? dot={(props) => {
??????????????? const { payload, cx, cy } = props;
??????????????? if (payload.milestone) {
????????????????? return (
??????????????????? <g>
????????????????????? <circle cx={cx} cy={cy} r={6} fill="#dc2626" />
????????????????????? <text
??????????????????????? x={cx}
??????????????????????? y={cy + 20}
??????????????????????? textAnchor="middle"
??????????????????????? fill="#374151"
style={{ fontSize: '10px' }}
                        transform={`rotate(45, ${cx}, ${cy + 20})`}
                      >
                        {payload.milestone}
                      </text>
                    </g>
                  );
                }
                return <circle cx={cx} cy={cy} r={3} fill="#2563eb" />;
              }}
            />
          </LineChart>
        </div>
      </CardContent>
    </Card>
  );
}        

Applications

·????? The use case demoed here can be used to Generate Summarized Infographics from multi image imput

·????? Extract information from images and combine with Tabular data or other infographics to generate any combined data structure like JSON or XML for downstream Analytics and consumption

·????? Textual Summary obtained from infographics maybe further converted to Audio for creating virtual Analytics Assistant personas

?

Acknowledgement

Dr. Pankaj Pansari ?for introducing me to Claude 3.5 and for this and other amazing use case ideas