Overview:
Apollo is an ambitious project that aims to embody a mix of technical expertise and architectural philosophy. Utilizing a well-defined tech stack that includes Python, Flask, and Vue with a rich admin interface from Element UI, the project showcases a clear separation between front and back-end processes. The architecture supports HTTP requests from the front-end to the back-end, which contributes to a cleaner and more maintainable codebase. As of now, it doesn’t utilize microservices, but it presents a straightforward deployment process that can serve as a foundation for future scalability.
The project is designed to build a web scrapping tool focused on fetching content from online forums. Key functionalities are outlined, revealing a roadmap that spans from task creation to user management and log monitoring. Each task is meticulously tracked, providing evolution insights and ensuring a robust user experience without compromising performance.
Features:
- Tech Stack: Built on Python, Flask, Vue, Element UI, MySQL, Redis, and Celery, providing a robust foundation for both back-end and front-end development.
- Modular Architecture: Front-end and back-end are clearly separated, enhancing code maintainability and encouraging clean coding practices.
- Web Scraping Tasks: Capable of creating scraping tasks based on post IDs, it extracts content, replies, and user info, making it highly functional for data retrieval.
- Results Visualization: Offers a user-friendly interface for viewing the outcomes of scraping tasks, enhancing user interaction and satisfaction.
- User Management: Incorporates login checks for user authorization, ensuring that non-registered users are restricted from accessing specific functionalities.
- Service Registration: Mimics a service registry with health checks, promoting reliability in task execution and management.
- Monitoring and Logging: Integrates with monitoring tools like Grafana and offers comprehensive logs of all API operations, assisting in performance and error tracking.
- Future Optimization Plans: Includes plans for implementing Elasticsearch for improved query performance and a focus on upgrading system capabilities to enhance overall performance.
