The method _assign_and_convert_leads() gets called when the CRM: Assign Leads cron is called.
The values in the list `weights` gets caluclated in a way that memebers with a lower
`lead_month`count` value gets a higher weight when randomizing assignement of a new lead. Assuming that the
max assignment per member is consistent (or default = 30).
Assume that each sale member belonging to any team has around 500 leads assigned to them the previous
month(`lead_month_count`) and `work_days` is set to `0.2`. The return value of the method
`_get_assignement_quota()` in this case would be 0 for every member, which causes the list `weights`
to get populated with just zeros. This raises a `ValueError: Total of weights must be greater than zero`.
Fix:
Make sure the minimum weight for each member is at least 1 and not 0.
opw-3171085
closesodoo/odoo#121908
X-original-commit: 15e2a1ebe2e64b752fb1c1d6bc422beb7d548b78
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Not entirely sure about TestAllocationRights. For TestEsEdiCommon
issue is quite obviously that it's inherited by tests which are
external, so when the `post_install_l10n` tag gets applied those tests
get run during "normal" l10n and they break.
closesodoo/odoo#98814
Related: odoo/enterprise#30825
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>
The performances tests in TestLeadAssignPerf have a margin for queries
because of some randomness in query counts.
Those margins avoid random failures but will hide small increments,
making the query count fail randomly in future build. This margin also
makes the update of query counts difficult.
This commit tries to identify the source of this randomness and proposes
some fixes.
Sources of randomness
=====================
Comparing different executions, the query can differ on two point:
1. During the flush, while updating the lead
1.1 the written user_id can vary (~1 additional query)
1.2 the written convert_date can changes (~4 additional queries)
1.3 the written date_last_stage_update can change (~1 additional query)
2. during _handle_salesmen_assignment
2.1 write/_message_auto_subscribe can change? (~1 additional query)
the point 1.2 and 1.3 can be easily reproduced, adding sleep,
especially a 0.1 sleep in convert_opportunity
Writing different values for date will lead to multiple execute
when flushing the records:
the orm cannot group records with different values
Proposed fixes
==============
A. Use `cr.now` instead of `fields.datetime.now`
Using the transaction time will avoid randomness linked to the change of
second during the transaction. This will fix 1.2 and 1.3
B. Sorting members
The members comes from a o2m and the order is not deterministic.
During the test with a subset of lead (10 instead of 100),
the two last members ('Martin Sales Manager' and 'Orteil Sales Own')
can come in different orders, leading to different user_id set on lead.
Instead of 4 different users, the lead where sometimes dispatch on
5 different users with this setup. (5 queries in flush() instead of 4)
This is fixed by adding a complete order (adding id) on crm.team.members
This will solve 1.1
C. The leads are also ordered by id if they have the same probability,
since it shouldn't hurt and make the search more deterministic.
2.1 was fixed by B or C or is not fixed, not sure anymore.
task-2796579
Part-of: odoo/odoo#85525
The test `test_assign_count` may fail randomly leading to the error:
```
self.assertEqual(self.sales_team_1_m3.lead_month_count, 14)
AssertionError: 12 != 14
```
Since BUNDLE_HOURS_DELAY is 0 (during tests), the domain becomes
`[('create_date', '<', fields.Datetime.now()]`
Depending of the time when the transaction starts, and the tests is
executed, the domain could match more or less records when executed one
second earlier or later. This is especially True for single module build
where there is less logic and the execution is faster. The domain may be
tested earlier and match less records.
This error can be reproduced deterministically by setting
`BUNDLE_HOURS_DELAY = 1/3600`
A fix could be to ignore ignore the create_date domain if the
BUNDLE_HOURS_DELAY is 0, we want to match all records in theory
(except if some of them are created in the future...)
The chosen solution is to use cr.now and use a <= instead.
Some assertions where also added to the test in order to identify why
the count was different and give a more precise message when it fails.
```
AssertionError: Lists differ: [] != ['TestLeadInitial_0000', 'TestLeadInitial_[97 chars]006']
Second list contains 6 additional elements.
First extra element 0:
'TestLeadInitial_0000'
- []
+ ['TestLeadInitial_0000',
+ 'TestLeadInitial_0001',
+ 'TestLeadInitial_0002',
+ 'TestLeadInitial_0004',
+ 'TestLeadInitial_0005',
+ 'TestLeadInitial_0006']
```
task-2796579
closesodoo/odoo#87107
X-original-commit: 4a5b68f05c57ba36560d43733dfd04b67c3297a8
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Seed is now fixed at beginning of each test to ensure random state is set
once for all and avoid random issues.
Task-2643740
X-original-commit: 85ebcaeaf909cd46743c20bf052360a65e60ea82
Part-of: odoo/odoo#79573
Some tests in this test class are leading to uncertain results and fails
randomly.
With this commit, the whole test class is deactivated until a fix is found.
closesodoo/odoo#76743
X-original-commit: 39545ba5b3c080ae3e651c4f9d25aa190582429a
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
Purpose is to avoid randomly failing tess depending on how leads are allocated
through teams. We achieve that purpose by
* creating more leads, easing their repartition through teams;
* adding a lower and upper bound test for team allocation to ensure weights
are globally ok;
* changing member assign domain to fit more closely test lead data;
We also improve test by using the ``crm.assignment.commit.bundle`` config
parameter that has an impact on lead duplicates removing and therefore
performances.
LINKS
Task ID-2528906
COM PR odoo/odoo#70756
Followup of Task ID-2489951 (assign process improvements)
Followup of odoo/odoo#70172 (smoother assign process)
X-original-commit: 8fdf97b6498adc0a2fb15e360ebbbaa1672534e5
Add a test ensuring ``assignment_optout`` works as intended. Salespersons
being opt-out from assign process should not receive any leads. This replaces
setting ``assignment_max`` to 0 (which is still a working trick). This commit
is a followup of odoo/odoo#68476 (done in a rush), and done in a specific
commit to avoid messing with forward-port.
Task ID-2497077
COM PR odoo/odoo#70664
X-original-commit: 813016e77b6abf9032059725a8725f693b132b58
PURPOSE
All unassigned leads should be assigned to teams as soon as possible to ease
lead analysis.
Purpose of assign thresholds is to ensure sales people receive at least this
amount of leads within 30 days, counting lost and won leads. Giving them leads
regularly is also one goal of automatic assign.
SPECIFICATIONS
Counting every lead whatever its state may lead to an inconvenient situation.
If a salesman always reaches its maximum every days he will always receive the
number of lead he got 30 days ago. For example if the salesman goes in holidays
for few days and set the max to 0 he receives no lead during his vacation.
Then after a few days of assign he reaches its maximum and receive 0 leads
for a few days. This leads to having windows of leads that repeat themselves
every 30 days.
* Solution: do not limit at maximum capacity anymore. Compensation is voided
if limit is achieved. However asked assignment is done. Salesman could
receive more than its max capacity but 30 days window ensure old leads
are regularly going out of count.
``assignment_max`` is now more a mean target of leads to be assigned during
a 30 days window than a real maximum capacity. Field is renamed accordingly.
LINKS
Task ID-2444908 (assign fixes)
Task ID-2489951 (assign process improvements)
COM PR odoo/odoo#70172
X-original-commit: 3d1841f26ce64f0106122e48750d2c653ea9790f
Co-authored-by: Thibault François <tfr@odoo.com>
Co-authored-by: Thibault Delavallée <tde@odoo.com>
PURPOSE
All unassigned leads should be assigned to teams as soon as possible to ease
lead analysis.
Purpose of assign thresholds is to ensure sales people receive at least this
amount of leads within 30 days, counting lost and won leads. Giving them leads
regularly is also one goal of automatic assign.
SPECIFICATIONS: TEAM ALLOCATION
Team assignment has to be updated as we may have team domains that overlap.
We therefore remove maximum number of leads to allocate to teams. Instead all
available unassigned leads are allocated within teams.
- Solution: assign all available leads and not a count based on team's
capacity. This notably reverts the main goal of odoo/odoo@6df2f0cfc0
(see odoo/odoo#48422)
This assignment process is done proportionally to the team capacity. It is
computed as the sum of each member's maximum assignment counter. This means
that with a team having twice as much sale capacity than another team sharing
the same domain: first team should receive about 2/3 of leads while the second
one should receive the remaining 1/3.
- Solution: assign lead one by one. Choose a team randomly using a weighted
random algorithm, based on team's members capacity.
SPECIFICATIONS: MEMBER ASSIGN
Counting every lead whatever its state may lead to an inconvenient situation.
Moreover salespersons may opt-out from assign by setting their max capacity
to 0, for example when going on holidays.
When doing that lead assignment is not smooth and getting back to a full
pipe may take several days. To solve that issue a compensation is added in
assignment quota done to sales people. Salespersons having few leads will
get a boost in assign as soon as they get back in assign process. When being
near maximum compensation is nearing 0 and daily quota is given.
PERFORMANCE
During team assignation, assigning lead one by one may cause performance issue.
Since PLS is computed at each flush, so we want to flush after a bunch of lead.
To solve that we obviously need to not commit at each assignation but
also avoid to search for duplicate at each assignation, that why the
search for duplicates is done at the beginning of the process and stored
in memory.
LINKS
Task ID-2444908 (assign fixes)
Task ID-2489951 (assign process improvements)
COM PR odoo/odoo#70172
X-original-commit: 83f72711e7affa8b0ad997b9ee262326e9577d54
Automatic assignment is performed on leads that are unassigned and not won
nor lost. This commit fixes the condition that should be based only on
stage being set to a stage with ``is_won`` being True.
Current code fails with won unassigned leads that are assigned even if it
should not. This happens rarely as won leads have probably been taken care
of. However having the right condition ensure filtering won and lost leads
is not correctly done.
Followup of odoo/odoo@5016b47a06 (see odoo/odoo#48422)
LINKS
Task ID-2444908 (assign fixes)
Task ID-2489951 (assign process improvements)
COM PR odoo/odoo#70172
X-original-commit: d9fe86ec2ca6459a34b31e65c0702d8a80a7ec61
Co-authored-by: Thibault François <tfr@odoo.com>
Co-authored-by: Thibault Delavallée <tde@odoo.com>
New leads are automatically assigned to sales team and salesman during assign
process. Some of those leads are duplicates of existing opportunities. If a
new lead is merged into an existing opportunity, the opportunity is the master
data and the lead is duplicated. However the opportunity should keep its sales
team and not be assigned to the team being under automatic assign.
Before this commit, the merge was forcing the crm team to the 'current' team
during auto assignment and thus it was changing the team of the opportunity.
In this commit we now assign team to lead to merge before the merge and do
not force the crm team during merge process.
LINKS
Task ID-2444908 (assign fixes)
Task ID-2489951 (assign process improvements)
COM PR odoo/odoo#70172
X-original-commit: 356a55053eb1cc49ae6eed0e58338cff08eddc42
PURPOSE
Goal of ``assignment_max`` is to make sure sales receives this amount of leads
within 30 days. Counting lost or won leads is therefore necessary to effectively
count all leads received those last 30 days.
SPECIFICATIONS
Change ``lead_month_count`` computation to take into account archived and won
leads. We consider now that goal of ``assignment_max`` is to make sure sales
people receive this amount of leads within the 30 days at max.
This allows to avoid archiving leads to get a new batch of leads to handle.
Also done leads should be counted as it is part of the month job.
LINKS
Task ID-2444908 (assign fixes)
Task ID-2489951 (assign process improvements)
COM PR odoo/odoo#70172
X-original-commit: 23feb1a467e0a441eb775be26c96af697d74506f
This commit contains two main preparatory changes in tests
* 1) lead_month_count will be changed to count lost lead as well. So
archiving leads before running the test doesn't reset the counter.
Lead needs to be unlinked in order to make tests reproducible;
* 2) assertMemberAssign checked the validity of the team domain. Normally
all lead assigned to the member should match the domain of its team.
This assert is not always verified since we force the team after a
merge of duplicated leads. Final merged lead may not match team domain
anymore as final properties depend on all merged leads.
Some tests are also added :
* add tests for master lead team_id and user_id values when merging leads
in assign process;
* add tests using email_normalized check in duplicate finding to ensure that
normalized emails are used for duplicates and not only exact match;
* add tests for won / lost inclusion in ``_get_lead_duplicates``, as well
as different between probability=100 and stage.is_won = True . This is
going to change in upcoming commits, hence some test to highlight it;
* add tests counting existing leads for assignment purpose;
* add tests for assignment_max maximum leads allocation as well as max=0
meaning salesperson is opt-outed from assign;
* add tests for default user / team / stage computation for leads;
* add tests about leads taken into account in assign process (currently
failing with won unassigned leads);
We also add some flush to ensure some fields notably related to probabilities
are flushed before counting queries.
LINKS
Task ID-2444908 (assign fixes)
Task ID-2489951 (assign process improvements)
COM PR odoo/odoo#70172
X-original-commit: bd3444085974f4e12287c2f5da4fb33308cd6180
Co-authored-by: Thibault François <tfr@odoo.com>
Co-authored-by: Thibault Delavallée <tde@odoo.com>
Purpose is to split assign tests related to results (leads being assigned,
with deduplication, compared to team and member domains) from tests related
to performance (query counters).
Assign process is a random process: randomizing teams leads to searching,
assigning and de-duplicating leads in various order. As a lot of search
are implied during assign process query counters may vary from run to run.
"Heavy" performance test included here ranged from 6K to 6.3K queries. Either
we set high counters maximum which makes those tests less useful. Either we
avoid random if possible which is what we decided to do by setting the seed
of random in tests.
SPECIFICATIONS
For results oriented tests remove query counters. Those tests should simply
ensure results of assign process, notably that even with random choice between
team and users we always have leads assigned with right domain and counts.
For performance oriented tests using query counters set random seed to have
a reproducible scenario and have less random in query counters.
LINKS
Task ID-2446759 (lead merge priority field management)
Task ID-2446883 (query counters fix)
PR odoo/odoo#65016
GLOBAL PURPOSE
Ability to have salesmen belonging to several sales team is a core requirement
of CRM. It is therefore moved from website_crm_score to crm along with cleaning
and behavior improvement. Automatic lead assignment is also moved and cleaned.
SPECIFICATIONS
Currently when assignment runs all free leads are allocated to teams. They
are then assigned to members according to their capacity. This means that
no lead is kept free. If assignment is performed on a subset of teams this
means that other teams have no free lead anymore.
During assignment evaluate which teams still need to receive leads. This is
based on team maximum capacity. We consider a team should receive twice its
capacity as leads. That way members will receive leads and can pick some leads
in team unassigned pool of leads.
e.g. a team has a total of capacity of 90 leads / month among its 3 members.
When running a daily assignment process, an equivalent of twice this running
time is assigned to members (see previous commits). It leads to assignment
of (90 / 30) * 2 = 6 leads. Twice this count is allocated to the team aka
12 leads. Remaining leads will be allocated in next assignment run or are
available for manual assign.
LINKS
Task ID-2086889 (main task)
Task ID-2357969 (scoring migration task)
Community PR odoo/odoo#48422
Enterprise PR odoo/enterprise#499
Upgrade PR odoo/upgrade#996
GLOBAL PURPOSE
Ability to have salesmen belonging to several sales team is a core requirement
of CRM. It is therefore moved from website_crm_score to crm along with cleaning
and behavior improvement. Automatic lead assignment is also moved and cleaned.
SPECIFICATIONS
Move sales team automatic lead assignment from website_crm_score (enterprise)
directly into sales_team. Some cleaning and renaming is performed while moving
the code as it is old fashioned.
Scoring feature is removed completely from lead assignment. Indeed we consider
PLS is a better feature and provides more accurate results.
Minimal scoring specific field on team is removed. Indeed it can also be
integrated directly into domains linked to a sales team as this is a field
directly available on lead using ``probability``. It allows to reduce fields
of team model and move all logic in the team's domain itself.
Overall behavior as been kept as close as possible from the original one.
Purpose is to avoid too much undesired changes between versions.
In this commit we also add tests for automatic lead assignment.
ASSIGNMENT PROCESS DESCRIPTION
1- LEAD TO TEAM ALLOCATION
Allocate leads to teams given by self. This method sets ``team_id`` field
on lead records that are unassigned (no team and no responsible). No
salesperson is assigned in this process. Its purpose is simply to allocate
leads within teams.
Heuristic of this method is the following:
* first we randomize all teams;
* then for each team
* find unassigned leads, aka leads being
* without team, without user -> not assigned;
* not in a won stage, and not having False/0 (lost) or 100 (won)
probability) -> live leads;
* if set, a delay after creation can be applied (see BUNDLE_HOURS_DELAY)
parameter explanations here below;
* keep only leads matching the team's assignment domain (empty means
everything);
* assign maximum BUNDLE_SIZE leads to the team, then move to the next team.
This is done to ensure every team will have leads enough to fill its
capacity based on its domain;
* when setting a team on leads, leads belonging to the current batch
are also merged. Purpose is to clean database and avoid assigning
duplicates to same or different teams;
* for all teams that still have capacity (aka: a search on unassigned
available leads with team domain still give results), do another
assignment round. Each round teams are randomized so that team order
is not always the same;
Note that leads are assigned in batch meaning a team could receive leads that
could better fit another team. However this heuristics is based on hypothesis
that team domains do not overlap. Indeed if a company has several teams they
will probably target separate market segments: country-based, customer type or
size, ... Having several teams using same assignment domain could lead to less
fairness in assignment process but this should not be the target use case of this
heuristic.
Leads are allocated by batch. This can be configured using a config parameter.
Batch size depends on cron frequency, lead pipeline size and members assignment
maximum. Finding an optimal heuristic for this parameter is not easy as it
depends on internal processes and organization. Higher batch size leads to
better performances when running automatic assignment. It can also give unfair
results if teams domain overlap or if pipeline is not big enough to fill all
teams capacity.
2- LEAD TO MEMBER ASSIGN
Main processing method to assign leads to sales team members. It also converts
them into opportunities. Its main purpose is therefore to distribute team
workload on its members based on their capacity.
Preparation
* prepare lead domain for each member. It is done using a logical AND with
team's domain and member's domain. Member domains further restricts team
domain;
* prepare a set of available leads for each member by searching for leads
matching domain with a sufficient limit to ensure all members will receive
leads;
* prepare a weighted population sample. Population are members that should
receive leads. Initial weight is the number of leads to assign to that
specific member. This is minimum value between
* remaining this month: assignment_max - number of lead already assigned
this month;
* days-based assignment: assignment_max with a ratio based on ``work_days``
* e.g. Michel Poilvache (max: 30 - currently assigned: 15) limit
for 2 work days: min(30-15, 30/15) -> 2 leads assigned
* e.g. Michel Tartopoil (max: 30 - currently assigned: 26) limit
for 10 work days: min(30-26, 30/3) -> 4 leads assigned
Assign process then follows the following heuristic
* take a weighted random choice in population;
* find first available (not yet assigned) lead in its lead set;
* if found:
* convert it into an opportunity and assign member as salesperson;
* lessen member's weight so that other members have an higher
probability of being picked up next;
* if not found: consider this member is out of assignment process,
remove it from population so that it is not picked up anymore;
Assignment is performed one lead at a time for fairness purpose. Indeed members
may have overlapping domains within a given team. To ensure some fairness in
process once a member receives a lead, a new choice is performed with updated
weights. This is not optimal from performance point of view but increases
probability leads are correctly distributed within the team.
LINKS
Task ID-2086889 (main task)
Task ID-2357969 (scoring migration task)
Community PR odoo/odoo#48422
Enterprise PR odoo/enterprise#499
Upgrade PR odoo/upgrade#996
adda-odoo