🕷️ Crawler Inspector

[Skip to main content](https://stackoverflow.com/questions/36746071/building-your-own-nlp-api#content) [Stack Overflow](https://stackoverflow.com/) 1. [About](https://stackoverflow.co/) 2. Products 3. [For Teams](https://stackoverflow.co/teams/) 1. [Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=stack-overflow-for-teams) 2. [Advertising Reach devs & technologists worldwide about your product, service or employer brand](https://stackoverflow.co/advertising/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=stack-overflow-advertising) 3. [Knowledge Solutions Data licensing offering for businesses to build and improve AI tools and models](https://stackoverflow.co/api-solutions/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=overflow-api) 4. [Labs The future of collective knowledge sharing](https://stackoverflow.co/labs/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=labs) 5. [About the company](https://stackoverflow.co/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=about-the-company) [Visit the blog](https://stackoverflow.blog/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=blog) 1. ### [current community](https://stackoverflow.com/) - [Stack Overflow](https://stackoverflow.com/) [help](https://stackoverflow.com/help) [chat](https://chat.stackoverflow.com/?tab=explore) - [Meta Stack Overflow](https://meta.stackoverflow.com/) ### your communities [Sign up](https://stackoverflow.com/users/signup?ssrc=site_switcher&returnurl=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F36746071%2Fbuilding-your-own-nlp-api) or [log in](https://stackoverflow.com/users/login?ssrc=site_switcher&returnurl=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F36746071%2Fbuilding-your-own-nlp-api) to customize your list. ### [more stack exchange communities](https://stackexchange.com/sites) [company blog](https://stackoverflow.blog/) 2. [Log in](https://stackoverflow.com/users/login?ssrc=head&returnurl=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F36746071%2Fbuilding-your-own-nlp-api) 3. [Sign up](https://stackoverflow.com/users/signup?ssrc=head&returnurl=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F36746071%2Fbuilding-your-own-nlp-api) 1. 1. [Home](https://stackoverflow.com/) 2. [Questions](https://stackoverflow.com/questions) 3. [AI Assist Labs](https://stackoverflow.ai/) 4. [Tags](https://stackoverflow.com/tags) 5. [Challenges](https://stackoverflow.com/beta/challenges) 6. [Chat](https://chat.stackoverflow.com/rooms/259507/stack-overflow-lobby) 7. [Articles](https://stackoverflow.blog/contributed?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=so-blog&utm_content=experiment-articles) 8. [Users](https://stackoverflow.com/users) 9. [Companies](https://stackoverflow.com/jobs/companies?so_medium=stackoverflow&so_source=SiteNav) 10. [Collectives]() 11. Communities for your favorite technologies. [Explore all Collectives](https://stackoverflow.com/collectives-all) 2. Teams  Ask questions, find answers and collaborate at work with Stack Overflow for Teams. [Try Teams for free](https://stackoverflowteams.com/teams/create/free/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=side-bar&utm_content=explore-teams) [Explore Teams](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=side-bar&utm_content=explore-teams) 3. [Teams]() 4. Ask questions, find answers and collaborate at work with Stack Overflow for Teams. [Explore Teams](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=side-bar&utm_content=explore-teams-compact) ##### Collectives™ on Stack Overflow Find centralized, trusted content and collaborate around the technologies you use most. [Learn more about Collectives](https://stackoverflow.com/collectives) **Teams** Q\&A for work Connect and share knowledge within a single location that is structured and easy to search. [Learn more about Teams](https://stackoverflow.co/teams/) # [Building your own NLP API](https://stackoverflow.com/questions/36746071/building-your-own-nlp-api) [Ask Question](https://stackoverflow.com/questions/ask) Asked 9 years, 6 months ago Modified [2 years, 5 months ago](https://stackoverflow.com/questions/36746071/building-your-own-nlp-api?lastactivity "2023-05-04 21:59:30Z") Viewed 3k times Part of [NLP](https://stackoverflow.com/collectives/nlp) Collective 5 I'm building a chatbot and I'm new to NLP. (api.ai & AlchemyAPI are too expensive for my use case. And wit.ai seems to be buggy and constantly changing at the moment.) For the NLP experts, how easily can I replicate their services locally? My vision so far (with node, but open to Python): - entity extraction via StanfordNER - intent via NodeNatural's LogisticRegressionClassifier - training UI with text and validate/invalidate buttons (any prebuilt tools for this?) Are entities and intents all I'll need for a chatbot? How good will NodeNatural/StanfordNER be compared to NLP-as-a-service? What headaches am I not seeing? - [python](https://stackoverflow.com/questions/tagged/python "show questions tagged 'python'") - [node.js](https://stackoverflow.com/questions/tagged/node.js "show questions tagged 'node.js'") - [nlp](https://stackoverflow.com/questions/tagged/nlp "show questions tagged 'nlp'") - [chatbot](https://stackoverflow.com/questions/tagged/chatbot "show questions tagged 'chatbot'") [Share](https://stackoverflow.com/q/36746071 "Short permalink to this question") [Improve this question](https://stackoverflow.com/posts/36746071/edit) Follow asked Apr 20, 2016 at 13:54 [](https://stackoverflow.com/users/377856/kyle-cureau) [Kyle Cureau](https://stackoverflow.com/users/377856/kyle-cureau)Kyle Cureau 19\.5k2323 gold badges7878 silver badges106106 bronze badges 2 - 1 You almost answered your own question: if those API you mention are too expensive, chances are they provide functionality that is not easy to implement (at least optimally). However, you probably do not need all of the functionality provided by them. Finally, why not use the open source Python NLP library, namely [nltk.org](http://www.nltk.org/) ?? user2321808 – user2321808 2016-04-20 14:03:44 +00:00 Commented Apr 20, 2016 at 14:03 - @William, yea I like nltk. But being new to chatbots and NLP in general, I'd like to know what I'm in for. My optimistic sentiment is "oh, ok I'll just use some nltk modules and be done by dinner!" I must be missing something, and I want to know what specifically. Kyle Cureau – [Kyle Cureau](https://stackoverflow.com/users/377856/kyle-cureau "19,476 reputation") 2016-04-20 14:20:34 +00:00 Commented Apr 20, 2016 at 14:20 [Add a comment](https://stackoverflow.com/questions/36746071/building-your-own-nlp-api "Use comments to ask for more information or suggest improvements. Avoid answering questions in comments.") \| ## 3 Answers 3 Sorted by: [Reset to default](https://stackoverflow.com/questions/36746071/building-your-own-nlp-api?answertab=scoredesc#tab-top) 3 You seems to have done your home work. Like you said,following things will help you along the way, - any classification algorithm for identifying the intent such as sklearn's LinearSVC or LogisticRegression Classifier - Any good NER tool such as StanfordNER or CRFsuite. CRFsuite has easy to use pthon wrapping called pycrfsuite. - A sentiment analysis tool for more human like conversations. its better if you go with python because python has lot of free libraries for the same. Only benefit of services such as wit.ai or api.ai is their heavily trained ready to use intents and models.You too will be able to achive similar accuracy if you are able to provide decent amount of training to your bot. It's better if you build on some existing opensource libraries rather than building everything from scratch. Please [check my opensource project](https://github.com/alfredfrancis/ai-chatbot-framework) on github for wit.ai/api.ai similar interface. Happy coding\! [Share](https://stackoverflow.com/a/42420022 "Short permalink to this answer") [Improve this answer](https://stackoverflow.com/posts/42420022/edit) Follow answered Feb 23, 2017 at 15:34 [](https://stackoverflow.com/users/1124056/alfred-francis) [Alfred Francis](https://stackoverflow.com/users/1124056/alfred-francis)Alfred Francis 45111 gold badge77 silver badges2020 bronze badges Sign up to request clarification or add additional context in comments. ## Comments Add a comment 0 Have a look at [Luis.ai](http://luis.ai/) (from Microsoft). It will help you build a Natural Langauge model that identifies intents. You can map intents to actions. It allows you to include pre-configured NLP models so that your bot can decipher text and return entities (names, time, geography, etc) and Luis supports many languages. You can of course add your own entities. And then you train it with utterances. And when users talk to your bot, those utterances are stored online and Luis.ai which will suggest that you add some to further refine your model via its suggest feature (this allows incremental improvements without coding). Now for the coding stuff... Visit Luis.ai and when you build your model (free) you can test it via the testing console here: <https://dev.projectoxford.ai/docs/services> and then you could use their Bot Framework; however I am building my own in JQuery/Bootstrap (have a look at this to see what I am building here <http://onlinebotbuilder.com>) and handling the json from the server via a post to the Cognitive Services API (which you would do in Python). Currently they allow 10k utterances per month; which for testing is plenty. Tip: When creating your model start with 2 intents and train your model to identify those. Keep it simple, and then expand scope as you go. Hope this helps. [Share](https://stackoverflow.com/a/41186654 "Short permalink to this answer") [Improve this answer](https://stackoverflow.com/posts/41186654/edit) Follow answered Dec 16, 2016 at 14:35 [](https://stackoverflow.com/users/264735/rob) [Rob](https://stackoverflow.com/users/264735/rob)Rob 1,21644 gold badges2323 silver badges4343 bronze badges ## Comments Add a comment 0 Two things to think about are: How are you planning on handling the generation side of things? Entity extraction and classification are going to be useful for the Natural language understanding (NLU) side of things, but generation can be tricky in itself. Another thing to think about is that the training and development of the pipeline of these models is often a separate problem form the deployment. The fact that you want to use node suggests that you already know about deploying software, I think. But remember that deploying large machine learning models in a pipeline can be complicated, and I suspect that these API's may offer neatly packaged pipelines for you. [Share](https://stackoverflow.com/a/54928184 "Short permalink to this answer") [Improve this answer](https://stackoverflow.com/posts/54928184/edit) Follow answered Feb 28, 2019 at 14:42 [](https://stackoverflow.com/users/5390664/adamits) [adamits](https://stackoverflow.com/users/5390664/adamits)adamits 9311 silver badge77 bronze badges ## Comments Add a comment Start asking to get answers Find the answer to your question by asking. [Ask question](https://stackoverflow.com/questions/ask) Explore related questions - [python](https://stackoverflow.com/questions/tagged/python "show questions tagged 'python'") - [node.js](https://stackoverflow.com/questions/tagged/node.js "show questions tagged 'node.js'") - [nlp](https://stackoverflow.com/questions/tagged/nlp "show questions tagged 'nlp'") - [chatbot](https://stackoverflow.com/questions/tagged/chatbot "show questions tagged 'chatbot'") See similar questions with these tags. [NLP Collective See more](https://stackoverflow.com/collectives/nlp) [This question is in a collective:](https://stackoverflow.com/collectives) a subcommunity defined by tags with relevant content and experts. - The Overflow Blog - [From multilingual semantic search to virtual assistants at Bosch Digital](https://stackoverflow.blog/2025/10/20/from-multilingual-semantic-search-to-virtual-assistants-at-bosch-digital/) - [Open source is giving you choices with your agent systems](https://stackoverflow.blog/2025/10/21/open-source-is-giving-you-choices-with-your-agent-systems/) - Featured on Meta - [A First Look: Stack Overflow Redesign](https://meta.stackexchange.com/questions/412992/a-first-look-stack-overflow-redesign) - [Community Asks Sprint Announcement - October/November 2025](https://meta.stackexchange.com/questions/413210/community-asks-sprint-announcement-october-november-2025) - [Policy: Generative AI (e.g., ChatGPT) is banned](https://meta.stackoverflow.com/questions/421831/policy-generative-ai-e-g-chatgpt-is-banned) - [Exploring new types of questions on Stack Overflow](https://meta.stackoverflow.com/questions/435121/exploring-new-types-of-questions-on-stack-overflow) #### Related [724](https://stackoverflow.com/questions/37986800/node-sass-couldnt-find-a-binding-for-your-current-environment "Question score (upvotes - downvotes)") [Node Sass couldn't find a binding for your current environment](https://stackoverflow.com/questions/37986800/node-sass-couldnt-find-a-binding-for-your-current-environment) [3](https://stackoverflow.com/questions/39472347/how-to-implement-bot-engine-like-wit-ai-for-on-an-on-premise-solution "Question score (upvotes - downvotes)") [How to implement BOT engine like WIT.AI for on an on-premise solution?](https://stackoverflow.com/questions/39472347/how-to-implement-bot-engine-like-wit-ai-for-on-an-on-premise-solution) [0](https://stackoverflow.com/questions/41211413/how-to-classify-a-sentence-based-on-a-few-templates "Question score (upvotes - downvotes)") [How to classify a sentence based on a few templates?](https://stackoverflow.com/questions/41211413/how-to-classify-a-sentence-based-on-a-few-templates) [0](https://stackoverflow.com/questions/44647233/how-to-extract-only-entities-not-intents-out-of-string-using-nlp "Question score (upvotes - downvotes)") [How to extract only entities not intents out of string using NLP?](https://stackoverflow.com/questions/44647233/how-to-extract-only-entities-not-intents-out-of-string-using-nlp) [1](https://stackoverflow.com/questions/62648586/training-data-format-with-spacy "Question score (upvotes - downvotes)") [Training Data Format with Spacy](https://stackoverflow.com/questions/62648586/training-data-format-with-spacy) [883](https://stackoverflow.com/questions/22519784/how-do-i-convert-an-existing-callback-api-to-promises "Question score (upvotes - downvotes)") [How do I convert an existing callback API to promises?](https://stackoverflow.com/questions/22519784/how-do-i-convert-an-existing-callback-api-to-promises) [783](https://stackoverflow.com/questions/47068709/your-cpu-supports-instructions-that-this-tensorflow-binary-was-not-compiled-to-u "Question score (upvotes - downvotes)") [Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2](https://stackoverflow.com/questions/47068709/your-cpu-supports-instructions-that-this-tensorflow-binary-was-not-compiled-to-u) [1](https://stackoverflow.com/questions/55001920/how-to-handle-two-entity-extraction-methods-in-nlp "Question score (upvotes - downvotes)") [How to handle two entity extraction methods in NLP](https://stackoverflow.com/questions/55001920/how-to-handle-two-entity-extraction-methods-in-nlp) [773](https://stackoverflow.com/questions/7067966/why-doesnt-adding-cors-headers-to-an-options-route-allow-browsers-to-access-my "Question score (upvotes - downvotes)") [Why doesn't adding CORS headers to an OPTIONS route allow browsers to access my API?](https://stackoverflow.com/questions/7067966/why-doesnt-adding-cors-headers-to-an-options-route-allow-browsers-to-access-my) #### [Hot Network Questions](https://stackexchange.com/questions?tab=hot) - [On the correct definition of ex-post variables](https://quant.stackexchange.com/questions/84139/on-the-correct-definition-of-ex-post-variables) - [Is Mage Hand physical and does it have a set size?](https://rpg.stackexchange.com/questions/216861/is-mage-hand-physical-and-does-it-have-a-set-size) - [Horizontally aligning bigwedges](https://tex.stackexchange.com/questions/752676/horizontally-aligning-bigwedges) - [Is it correct English to say "she is at her school break"?](https://ell.stackexchange.com/questions/369904/is-it-correct-english-to-say-she-is-at-her-school-break) - [Why can't a body move faster than light?](https://physics.stackexchange.com/questions/861155/why-cant-a-body-move-faster-than-light) - [Help w/ "posix-extended" regex for 'find'](https://unix.stackexchange.com/questions/800624/help-w-posix-extended-regex-for-find) - [What did Bertrand Russell mean by “reverence with the second”? Why does reverence interfere “with the second”?](https://philosophy.stackexchange.com/questions/131282/what-did-bertrand-russell-mean-by-reverence-with-the-second-why-does-reverenc) - [Coniferous tree with flat needle pattern](https://biology.stackexchange.com/questions/118039/coniferous-tree-with-flat-needle-pattern) - [Probability of friends having birthdays on different days](https://math.stackexchange.com/questions/5103035/probability-of-friends-having-birthdays-on-different-days) - [How to use zmv to add left padding to a number in a filename?](https://unix.stackexchange.com/questions/800629/how-to-use-zmv-to-add-left-padding-to-a-number-in-a-filename) - [How do I get this screw thing off my sliding door?](https://diy.stackexchange.com/questions/323665/how-do-i-get-this-screw-thing-off-my-sliding-door) - [How to release unused space?](https://dba.stackexchange.com/questions/348044/how-to-release-unused-space) - [How do I, as a born again Christian, properly follow Torah as a Messianic Christian?](https://christianity.stackexchange.com/questions/109075/how-do-i-as-a-born-again-christian-properly-follow-torah-as-a-messianic-christ) - [Have post-ruin dead bodies ever been found in actual ruins?](https://history.stackexchange.com/questions/79547/have-post-ruin-dead-bodies-ever-been-found-in-actual-ruins) - [Why do I need \\n when searching and \\r when replacing?](https://vi.stackexchange.com/questions/47292/why-do-i-need-n-when-searching-and-r-when-replacing) - [What is and isn't an indie game?](https://rpg.stackexchange.com/questions/216843/what-is-and-isnt-an-indie-game) - [Yuzzle's Puzzle Set \#1](https://puzzling.stackexchange.com/questions/133653/yuzzles-puzzle-set-1) - [What was the minimum amount of addressable memory? When and why did computers become byte-addressable?](https://retrocomputing.stackexchange.com/questions/32173/what-was-the-minimum-amount-of-addressable-memory-when-and-why-did-computers-be) - [Validate DNA Sequences with Telomeres](https://codegolf.stackexchange.com/questions/284174/validate-dna-sequences-with-telomeres) - [How can I reduce high CO₂ levels in my bedroom during winter without significant heat loss?](https://diy.stackexchange.com/questions/323616/how-can-i-reduce-high-co%E2%82%82-levels-in-my-bedroom-during-winter-without-significant) - [Is there a distinction between colors (secondary qualities) and shape/extension/solidity (primary qualities)?](https://philosophy.stackexchange.com/questions/131289/is-there-a-distinction-between-colors-secondary-qualities-and-shape-extension) - [Where to find the stairs in SUMMERHOUSE?](https://gaming.stackexchange.com/questions/413715/where-to-find-the-stairs-in-summerhouse) - [Sci-fi movie with fleshy game consoles that plug into the spine?](https://movies.stackexchange.com/questions/128615/sci-fi-movie-with-fleshy-game-consoles-that-plug-into-the-spine) - [What am I doing wrong with these wildcards in Microsoft Word?](https://superuser.com/questions/1926971/what-am-i-doing-wrong-with-these-wildcards-in-microsoft-word) [more hot questions](https://stackoverflow.com/questions/36746071/building-your-own-nlp-api) [Question feed](https://stackoverflow.com/feeds/question/36746071 "Feed of this question and its answers") # Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader.  default ##### [Stack Overflow](https://stackoverflow.com/) - [Questions](https://stackoverflow.com/questions) - [Help](https://stackoverflow.com/help) - [Chat](https://chat.stackoverflow.com/?tab=explore) ##### [Products](https://stackoverflow.co/) - [Teams](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=footer&utm_content=teams) - [Advertising](https://stackoverflow.co/advertising/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=footer&utm_content=advertising) - [Talent](https://stackoverflow.co/advertising/employer-branding/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=footer&utm_content=talent) ##### [Company](https://stackoverflow.co/) - [About](https://stackoverflow.co/) - [Press](https://stackoverflow.co/company/press/) - [Work Here](https://stackoverflow.co/company/work-here/) - [Legal](https://stackoverflow.com/legal) - [Privacy Policy](https://stackoverflow.com/legal/privacy-policy) - [Terms of Service](https://stackoverflow.com/legal/terms-of-service/public) - [Contact Us](https://stackoverflow.com/contact) - Cookie Settings - [Cookie Policy](https://policies.stackoverflow.co/stack-overflow/cookie-policy) ##### [Stack Exchange Network](https://stackexchange.com/) - [Technology](https://stackexchange.com/sites#technology) - [Culture & recreation](https://stackexchange.com/sites#culturerecreation) - [Life & arts](https://stackexchange.com/sites#lifearts) - [Science](https://stackexchange.com/sites#science) - [Professional](https://stackexchange.com/sites#professional) - [Business](https://stackexchange.com/sites#business) - [API](https://api.stackexchange.com/) - [Data](https://data.stackexchange.com/) - [Blog](https://stackoverflow.blog/?blb=1) - [Facebook](https://www.facebook.com/officialstackoverflow/) - [Twitter](https://twitter.com/stackoverflow) - [LinkedIn](https://linkedin.com/company/stack-overflow) - [Instagram](https://www.instagram.com/thestackoverflow) Site design / logo © 2025 Stack Exchange Inc; user contributions licensed under [CC BY-SA](https://stackoverflow.com/help/licensing) . rev 2025.10.21.35661