LLMs meet Network Analysis
AI Market Trends Analysis with LLM and Network Analysis
In class & Final Assignment
AI Technology Market Analysis Assignment
Group Project Using NLP and Network Analysis
Overview
Conduct a comprehensive analysis of AI technology markets by combining Natural Language Processing (NLP) and Network Analysis techniques. Use either the provided dataset or identify suitable alternative data sources that enable meaningful insights into AI market dynamics.
Core Requirements
Data Processing with LLMs
- Implement local or cloud-based Large Language Models (LLMs) to:
- Extract and structure relevant market data
- Identify network relationships between entities
- Perform named entity recognition and extraction
- Transform unstructured text into analyzable formats
Network Analysis
- Design and construct meaningful networks from the extracted data
- Implement bi-partite network analysis and corresponding projections
- Calculate and interpret key network metrics:
- Various centrality measures
- Network structure indicators
- Community detection (if applicable)
- Provide clear interpretation of network analysis results
Text Classification
Select and implement one of these approaches:
- LLM-based classification system
- Few-shot learning implementation using SetFit
- Traditional NLP classification methods (using existing or synthetic training data)
Optional Extensions
Topic Modeling
- Leverage LLMs to extract and categorize key themes and topics
- Apply BERTopic for advanced topic modeling
- Create clear and insightful visualizations of:
- Topic distributions
- Theme relationships
- Temporal patterns (if applicable)
Deliverables
Analysis Notebooks
- Well-documented Jupyter notebooks containing:
- Complete analysis pipeline
- Clear code documentation
- Inline result interpretation
- Reproducible implementation
Executive Summary
- Concise PDF slide deck (max 6 slides) including:
- Problem statement and approach
- Key findings and insights
- Visual representation of critical results