Speech tagging using Maximum Entropy models

Speech tagging using Maximum Entropy models

Francis Kurupacheril ?? Francis Kurupacheril ??

Francis Kurupacheril ??

Senior Product Management Professional

发布日期: 2022年10月25日
+ 关注

Maximum entropy modeling is a framework for integrating information from many heterogeneous information sources for classification.?The data for a?classification problem is described as a large number of features.?These features can be quite complex and allow the experimenter to make use of prior knowledge about what types of informations are expected to be important for classification. Each feature corresponds to a constraint on the model.?We then compute the maximum entropy model (maxent), the model with the maximum entropy of all the models that satisfy the constraints.?

The precision and recall accuracy figures for programs using maxent models are the state of the art on tasks like part of speech tagging, sentence detection, prepositional phrase attachment, and named entity recognition.

POS tagging is a specific use case that maxent models perform by combining several heterogenous features in a probabilistic network. In all NLP applications, the features are not independent and since their independence cannot be assumed, their 'word level' interdependence is quite pronounced. This allows for the Maximum entropy classifier to be used, because the characteristics are interdependent words. In the figure below, all the features (preceding and succeeding words, preceding and succeeding POS tags) around the W4 word "vehicles" is used in the model.

No alt text provided for this image

MaxEnt is log-linear or exponential algorithm and is depicted as extracting features, multiplying them by respective weights, and adding them up, then using this sum as an exponent:

No alt text provided for this image
Francis' ML and NLP notes Francis' ML and NLP notes

Francis' ML and NLP notes

812 位关注者

订阅

要查看或添加评论，请登录

Francis Kurupacheril ??的更多文章

  • Compilation of RAG Benchmarks with examples
    2024年8月15日

    Compilation of RAG Benchmarks with examples

    Let's explore practical examples for a few of the key RAG evaluation metrics and how they might be applied in…

    2 条评论
  • LLM's on your desktop
    2024年4月9日

    LLM's on your desktop

    Running large language models (LLMs) on a laptop or desktop introduces several complexities: ?First, the computational…

  • Open Source LLM's
    2024年3月31日

    Open Source LLM's

    Curious about the landscape of open-source Large Language Models (LLMs), including their features and licenses? Below…

    1 条评论
  • Decoding GenAI Leaderboards and LLM Standouts
    2024年3月28日

    Decoding GenAI Leaderboards and LLM Standouts

    The Generative AI (GenAI) landscape thrives on constant innovation. Large Language Models (LLMs) are pushing the…

    1 条评论
  • RAG (Retrieval Augmented Generation) with LLM's
    2023年10月26日

    RAG (Retrieval Augmented Generation) with LLM's

    A Retrieval-Augmented Generation (RAG) system integrated with a Large Language Model (LLM) operates in a two-step…

    2 条评论
  • Hallucination
    2023年4月21日

    Hallucination

    LLMs (Large Language Models), such as GPT-3 and BERT, are powerful models that have revolutionized the field of natural…

  • Pros and Cons of large language models
    2022年12月30日

    Pros and Cons of large language models

    Large language models have garnered significant attention in recent years due to their impressive performance on a wide…

    1 条评论
  • Named Entity Recognition using CRF's
    2022年11月22日

    Named Entity Recognition using CRF's

    Conditional Random Field (CRF). Conditional Random Field is a probabilistic graphical model that has a wide range of…

  • Support Vector Machines in NLP
    2022年9月24日

    Support Vector Machines in NLP

    "Support Vector Machine” (SVM) is a supervised machine learning algorithm that can be used for both classification or…

  • Bayesian Networks in NLP
    2022年8月25日

    Bayesian Networks in NLP

    A Bayesian network is a joint probability distribution of a set of random variables with a possible mutual causal…

社区洞察

其他会员也浏览了