The purpose is to ease the onboarding for the default CRM settings.
Now, we do not include any field by default and tags become an option
in the choice of fields apply to the computation.
task-2196200
closesodoo/odoo#48124
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose
=======
Get rid of the cron.
Always have an up to date frequency table to compute lead probability.
Clean PLS code and ensure to remove unnecessary code
Specifications
==============
Disable the cron:
- Increment won and lost count for lead PLS params from frequency table at
each won/lost
- The complete set of leads won't be recomputed anymore every day
- A lead probability will only be recomputed if one of the PLS parameter will
be modified (stage_id, team_id + the ones checked in settings)
- No more onboarding phase : as frequency table is always up to date and only
the modified leads are recomputed, we can drop this. (So even on onboarding,
winning or losing a lead won't impact other leads, except the ones we modify
manually afterwards)
- Add an button to force the recompute of all the lead probabilities
(kind of reset) in the settings.
The cron has been disabled by default.
Users can know choose to reactivate it if they want to continue updating their
lead probabilities on a daily basis.
Code have been reorganised for better structure and understanding.
Some code parts have been factorised to be usable in both Live Increment and
Full Rebuild mode.
In this commit, we also update probabilities using only computes
Before this commit, _update_probability was called during write and create,
to ensure both probability and automated_probability were correct. That was
necessary because of onchanges and readonly field (automated_probability
was not given to the write method while saving after a change on the lead).
After this commit, we use the force_save tag on the readonly field
automated_probability. So we ensure that write always receives the new
automated_probability value and everything can be handled directly by the
compute. Also, we force the lead to start in automated probability mode.
So when creating a new lead, both probability and automated probability are
computed and aligned.
Finally, when winning a lead, we ensure that both probabilities are aligned
(=100), so that if user take the lead back to a previous stage, the
probabilities are correctly recomputed, and the lead will not be stuck to
100%.
Also, this commit applies some fix and minor changes listed here under:
- Ensure to have always positive frequencies
- Fix lost counts for team not set in pls computation
- optimise set_won : write only once per values
Finally, include tag in frequency table even if less than 50 occurences:
Due to refactoring in which we increment frequency table at each won/lost
instead of rebuilding the frequency table each day, the tags are never added
in frequency table.
This is due to the 50 occurence conditions that avoid to create the frequency
if we have less than 50 won or lost for a tag.
After this commit, we include the tag_ids in the frequency table, no matter
the number of occurences. But, during the PLS computation, we skip the tag
frequencies if the number of occurences for this tag is under 50.
This allows to have a complete vision on the tag frequencies, even for
small number of occurences.
Task ID : 2254543
X-original-commit: cd291b79eb2d2df80899867263ec71438ab8fe87
* Code cleanup
* Avoid a safe evaluation of the field value when loading those records.
closesodoo/odoo#44883
Related: odoo/enterprise#8283
Signed-off-by: Victor Feyens (vfe) <vfe@odoo.com>
When there are only few leads, marking a lead as won or lost changes
a lot the won/lost repartition. For onboarding purpose, the cron is launched
if the threshold PLS config param is not reached.
When the number of lead in the database reaches the PLS threshold,
the config parameter is set to 0 to avoid the useless check
of the number of lead in the dabase in the future
Task ID: 2044539
PR #37413
This commit
- removes probability default value in order to activate
Bayes predictive lead scoring by default
- set the default pls start date to today - 8 days
to include all demo data in PLS computation
Task ID: 2044539
PR #37413
This commit applies multiples things:
- Fix PLS settings that were not loaded correctly (was not using technical field names)
- Add data for PLS settings to check all optional fields and set the start date.
- use null value instead of empty for phone and email state fields to lessen the DB storage
- set pls start date required
Task ID: 2044539
PR #37413
Since introduction of website_visitor,
we don't need to identifiy the lead with a lead_id cookie.
The visitor is identified and if a lead is created with the
contact form, the lead is created and linked to the visitor.
To avoid losing data, each time a visitor send a contact form,
it creates a new lead. All those leads are linked between them
via the website_visitor.
As the website_visitor already has the lang set on itself,
there is no reason to keep the lang_id of crm_lead at enterprise
side. So that lang_id on crm.lead has been moved to community.
This commit is directly linked to another one in enterprise
See https://github.com/odoo/enterprise/pull/4834 to get more details.
Task ID : 2028059
PR #34624
This new Predictive Lead Scoring (aka PLS) replaces the stage based probability mechanism.
It uses Naive Bayes probability computation theorem.
The lead probability is now computated automatically based on multiple criteria (= frequency fields).
Some are mandatory and will always be used, some are optional and can be activated
in the settings.
- Mandatory fields :
- team_id
- stage_id
- tag_ids
- Optional fields :
- email_state (validity of the email_from field: correct / incorrect or empty if email_from is empty)
- phone_state (validity of the phone field !! not the mobile one !!: correct / incorrect or empty if phone is empty)
- country_id
- state_id
- source_id (UTM Source)
PLS uses a start date to consider only the lead created after that date to generate the frequency table
and to recompute the lead probability. This date is also configurable in the CRM settings.
Default start date is date.todays() (at module install).
The settings for PLS are NOT company related :
PLS Fields = Many2many to a model that store only the fields allowed for PLS computation.
PLS Start Date = Date
As Date and Many2many fields are not allowed for config_parameters,
Char fields are used to store the configuration
and Date and m2m fields are used to ease the configuration by the user in the config panel.
Date and m2m fields are both computed fields based on their corresponding Char fields.
Char field for Date store a strigified date.
char field for Many2many store a comma separated string which is a list of activated fields.
Each mandatory field is used in a specific way
- Team_id : the frequency table is split for every team_id, plus once for leads that don't have a team_id.
Considering Lead A from Team 1 and Lead B from Team 2.
Even if Lead A and Lead B have the exact same attributes (country, phone, email,stage, ...),
their probability will most likely not be the same.
That's because each teams compute their leads' probability based on their own won/lost leads.
- Stage_id : if no optional fields are activated and if there is no tag on the lead,
this will make PLS work quite the same way as it was working with removed stage based probability,
as only the stage will count in the computation.
But here, instead of fixing manually the probability for each stage,
this stage related probability is automatically computed based on past experience (won and lost leads).
- Tag_ids : each tag is considered separatelly, as if the lead had only one tag.
(no link between tag on same lead is made). To avoid that a tag take to much importance if his subset is too
small, we include the tag frequencies in the frequency table only if at least 50 won or lost leads had this tag.
The frequency table is a table where all won and lost leads are aggregated by fields
(mandatory or optional if activated). For each team_id / field couple, we store the number of
won and lost leads that has that fields values.
For example: Lead A is assigned to team 1 and client comes from France and has been won.
If we consider only this lead A, the frequency table will looks like :
id | variable | value | won_count | lost_count | team_id
--------+-------------+-------+-----------+------------+---------
1 country_id FR 1 0 1
To computed the probability of a lead, we get all the records for the frequency table
that match all fields (per team_id) and we compute a won score and a lost score.
The probability is computed and normalized based on those scores
- P = S(Won) / (S(Won) + S(Lost))
Considering two variables A and B :
- S(Won) ∝ P(A∩B | Won)*P(Won) = P(A|Won) * P(B|Won) * P(Won)
- S(Lost) ∝ P(A∩B | Lost)*P(Lost) = P(A|Lost) * P(B|Lost) * P(Lost)
To overcome the 'zero frequency problem',
we do not start at 0 for the won or lost count for each variable / value combination.
It suggested to start at 1, but to avoid that a small subset take to much weight compared to
a larger one, we start at 0.1.
For example :
If we start at 1 : If team 1 has few records from France,
it can get high probability to win even if every lead where lost.
variable | value | won_count | lost_count | team_id | Probability
-------------+-------+-----------+------------+---------+-------------
country_id FR 1 7 1 0.125
country_id FR 198 7306 2 0.0263
If we start at 0.1 :
variable | value | won_count | lost_count | team_id | Probability
-------------+-------+-----------+------------+---------+-------------
country_id FR 0.1 6.1 1 0.0161
country_id FR 197.1 7305.1 2 0.0263
The more we have records for each couple, the more the computation become precise.
The allow the user to decide himself which probability should be set on the lead,
a manual probabiltiy system has been implemented using 2 stored fields + 1 computed field :
- probability (stored): Probability set on the lead, can be set manually in the lead form.
- automated_probability (stored): Probability computed by PLS.
- is_automated_probability (computed): If both previous fields are equal, the probability is considered
as automatic. and each time the automated_probability is updated, the probability is aligned.
If both are not equal, the probability is considered as manual. The automated_probability is still
updated but the probability is not aligned. The user can reset the probability in automatic mode
by clicking on the estimated probability displayed in edit mode in the lead form.
probability and automated_probability are not computed fields as they are manually computed at write and create.
The computation is triggered if one of the activated frequency fields is modified or set.
To trigger also the computation when modifying the lead values on the lead views (form and kanban),
_onchange methods have been added for each frequency field.
Also, a cron has been added to run every day to rebuild the frequency table and to recompute all active
and pending leads (not won nor lost) probability.
Task ID : 1925439
PR #33589
remove rerun cron on action lost and won
Jérémy Hennecart