Engineering an Intelligent Essay Scoring and Feedback System: An
Experience Report
- URL: http://arxiv.org/abs/2103.13590v1
- Date: Thu, 25 Mar 2021 03:46:05 GMT
- Title: Engineering an Intelligent Essay Scoring and Feedback System: An
Experience Report
- Authors: Akriti Chadda, Kelly Song, Raman Chandrasekar, Ian Gorton
- Abstract summary: We describe an exploratory system for assessing the quality of essays supplied by customers of a specialized recruitment support service.
The problem domain is challenging because the open-ended customer-supplied source text has considerable scope for ambiguity and error.
There is also a need to incorporate specialized business domain knowledge into the intelligent processing systems.
- Score: 1.5168188294440734
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Artificial Intelligence (AI) / Machine Learning (ML)-based systems are widely
sought-after commercial solutions that can automate and augment core business
services. Intelligent systems can improve the quality of services offered and
support scalability through automation. In this paper we describe our
experience in engineering an exploratory system for assessing the quality of
essays supplied by customers of a specialized recruitment support service. The
problem domain is challenging because the open-ended customer-supplied source
text has considerable scope for ambiguity and error, making models for analysis
hard to build. There is also a need to incorporate specialized business domain
knowledge into the intelligent processing systems. To address these challenges,
we experimented with and exploited a number of cloud-based machine learning
models and composed them into an application-specific processing pipeline. This
design allows for modification of the underlying algorithms as more data and
improved techniques become available. We describe our design, and the main
challenges we faced, namely keeping a check on the quality control of the
models, testing the software and deploying the computationally expensive ML
models on the cloud.
Related papers
- Dealing with Data for RE: Mitigating Challenges while using NLP and
Generative AI [2.9189409618561966]
Book chapter explores the evolving landscape of Software Engineering in general, and Requirements Engineering (RE) in particular.
We discuss challenges that arise while integrating Natural Language Processing (NLP) and generative AI into enterprise-critical software systems.
Book provides practical insights, solutions, and examples to equip readers with the knowledge and tools necessary.
arXiv Detail & Related papers (2024-02-26T19:19:47Z) - DistALANER: Distantly Supervised Active Learning Augmented Named Entity Recognition in the Open Source Software Ecosystem [4.368725325557961]
This paper proposes a novel named entity recognition (NER) technique specifically tailored for the open-source software systems.
Our approach aims to address the scarcity of annotated software data by employing a comprehensive two-step distantly supervised annotation process.
It is noteworthy that our model significantly outperforms the state-of-the-art LLMs by a substantial margin.
arXiv Detail & Related papers (2024-02-25T17:40:49Z) - Machine Learning Insides OptVerse AI Solver: Design Principles and
Applications [74.67495900436728]
We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver.
We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem.
We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
arXiv Detail & Related papers (2024-01-11T15:02:15Z) - Benchmarking Automated Machine Learning Methods for Price Forecasting
Applications [58.720142291102135]
We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions.
Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part.
We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
arXiv Detail & Related papers (2023-04-28T10:27:38Z) - AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities
and Challenges [60.56413461109281]
Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes.
We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful.
We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
arXiv Detail & Related papers (2023-04-10T15:38:12Z) - Design Automation for Fast, Lightweight, and Effective Deep Learning
Models: A Survey [53.258091735278875]
This survey covers studies of design automation techniques for deep learning models targeting edge computing.
It offers an overview and comparison of key metrics that are used commonly to quantify the proficiency of models in terms of effectiveness, lightness, and computational costs.
The survey proceeds to cover three categories of the state-of-the-art of deep model design automation techniques.
arXiv Detail & Related papers (2022-08-22T12:12:43Z) - Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering.
In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
arXiv Detail & Related papers (2022-03-06T10:12:56Z) - Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end.
We have developed a proven systems engineering approach for machine learning development and deployment.
Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
arXiv Detail & Related papers (2021-01-11T15:54:48Z) - Quality Management of Machine Learning Systems [0.0]
Artificial Intelligence (AI) has become a part of our daily lives due to major advances in Machine Learning (ML) techniques.
For business/mission-critical systems, serious concerns about reliability and maintainability of AI applications remain.
This paper presents a view of a holistic quality management framework for ML applications based on the current advances.
arXiv Detail & Related papers (2020-06-16T21:34:44Z) - Engineering AI Systems: A Research Agenda [9.84673609667263]
We provide a conceptualization of the typical evolution patterns that companies experience when employing machine learning.
The main contribution of the paper is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions.
arXiv Detail & Related papers (2020-01-16T20:29:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.